Hybrid victim cache and write miss buffer with fence operation

ABSTRACT

A caching system including a first sub-cache, and a second sub-cache, coupled in parallel with the first cache, for storing cache data evicted from the first sub-cache and write-memory commands that are not cached in the first sub-cache, and wherein the second sub-cache includes: color tag bits configured to store an indication that a corresponding cache line of the second sub-cache storing write miss data is associated with a color tag, and an eviction controller configured to evict cache lines of the second sub-cache storing write-miss data based on the color tag associated with the cache line.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Application No.62/852,494, filed May 24, 2019, titled “Data Caches and Related Systemsand Methods,” which is hereby incorporated herein by reference in itsentirety.

BACKGROUND

In a multi-core coherent system, multiple processor and systemcomponents share the same memory resources, such as on-chip and off-chipmemories. Memory caches (e.g., caches) typically are an amount ofhigh-speed memory located operationally near (e.g., dose to) aprocessor. A cache is more operationally nearer to a processor based onlatency of the cache, that is, one many processor dock cycles for thecache to fulfill a memory request. Generally, cache memory closest to aprocessor includes a level 1 (L1) cache that is often directly on a diewith the processor. Many processors also include a larger level 2 (L2)cache. This L2 cache is generally slower than the L1 cache but may stillbe on the die with the processor cores. The L2 cache may be a perprocessor core cache or shared across multiple cores. Often, a larger,slower L3 cache, either on die, as a separate component, or anotherportion of a system on a chip (SoC) is also available to the processorcores.

Memory systems such as caches can be susceptible to data corruption, forexample, due to electronic or magnetic interference from cosmic rays,solar particles, or malicious memory accesses. As processors areincreasingly used in critical and/or other fault-intolerant systems,such as self-driving vehicles and autonomous systems, techniques toprotect memory systems from data corruption are increasingly beingapplied to the memory systems. One such technique is the use of errorcorrecting codes (ECC) to detect and correct memory corruption.Implementing ECC in high speed cache memory is challenging as ECC canintroduce additional timing overhead that needs to be accounted for. Forexample, a high speed cache memory system may have a five stage memorypipeline for determining whether a memory address being accessed is inthe cache and retrieving the contents of the cache memory, Each stagemay take one clock cycle, which at GHz, is about one nanosecond. Errorchecking the contents of the cache memory can substantially take up afull clock cycle. What is needed are techniques for increasing cacheperformance for fault tolerant caches.

SUMMARY

This disclosure relates to a caching system. More particularly, but notby way of limitation, aspects of the present disclosure relate to acaching system including a first sub-cache and a second sub-cache inparallel with the first sub-cache, wherein the second sub-cache includesa set of cache lines, line type bits configured to store an indicationthat a corresponding cache line of the set of cache lines is configuredto store write-miss data, and an eviction controller configured to flushstored write-miss data based on the line type bits.

Another aspect of the present disclosure relates to a method for cachingdata including receiving, by a caching system, a write memory requestfor a memory address, determining, by a first sub-cache of the cachingsystem, that the memory address is not cached in the first sub-cache,determining, by second sub-cache of the caching system, that the memoryaddress is not cached in the second sub-cache, storing data associatedwith the write memory request in the second sub-cache, storing, in aline type bit of the second sub-cache, an indication that the storeddata corresponds to a write-miss, and flushing the stored data based onthe indication.

Another aspect of the present disclosure relates to a device including afirst sub-cache, and a second sub-cache in parallel with the firstsub-cache; wherein the second sub-cache includes a set of cache lines,line type bits configured store an indication that a corresponding cacheline of the set of cache lines is configured to store write-miss data,and an eviction controller configured to flush stored write-miss databased on the line type bits.

Another aspect of the present disclosure relate to a caching systemincluding a first sub-cache and a second sub-cache in parallel with thefirst sub-cache, wherein the second sub-cache includes: a set of cachelines, line type bits configured to store an indication that acorresponding line of the set of cache lines is configured to storewrite-miss data, and an eviction controller configured to evict a cacheline of the second sub-cache storing write-miss data based on anindication that the cache line has been fully written.

Another aspect of the present disclosure relates to a method for cachingdata, including receiving, by a caching system, a write memory requestfor a memory address, determining, by a first sub-cache of the cachingsystem, that the memory address is not cached in the first sub-cache,determining, by second sub-cache of the caching system, that the memoryaddress is not cached in the second sub-cache, storing data associatedwith the write memory request in the second sub-cache, storing, in aline type bit of the second sub-cache, an indication that the storeddata corresponds to a write-miss, and evicting a cache line of thesecond sub-cache storing the write-miss based on an indication that thecache line has been fully written.

Another aspect of the present disclosure relates to a device including aprocessor, a first sub-cache, and a second sub-cache in parallel withthe first sub-cache, wherein the second sub-cache includes: a set ofcache lines, line type bits configured to store an indication that acorresponding line of the set of cache lines is configured to storewrite-miss data, and an eviction controller configured to evict a cacheline of the second sub-cache storing write-miss data based on anindication that the cache line has been fully written.

Another aspect of the present disclosure relates to a caching systemincluding a first sub-cache, and a second sub-cache, coupled in parallelwith the first cache, for storing cache data evicted from the firstsub-cache and write-memory commands that are not cached in the firstsub-cache, and wherein the second sub-cache includes: color tag bitsconfigured to store an indication that a corresponding cache line of thesecond sub-cache storing write miss data is associated with a color tag,and an eviction controller configured to evict cache lines of the secondsub-cache storing write-miss data based on the color tag associated withthe cache line.

Another aspect of the present disclosure relates to a method for cachingdata, including receiving, by a caching system, a write memory commandfor a memory address, determining, by a first sub-cache of the cachingsystem, that the memory address is not cached in the first sub-cache,determining, by second sub-cache of the caching system, that the memoryaddress is not cached in the second sub-cache, wherein the secondsub-cache is configured to store, in parallel with the first sub-cache,cache data evicted from the first sub-cache and write-memory commandsthat are not cached in the first sub-cache, storing data associated withthe write memory command in the second sub-cache, storing, in the secondsub-cache, a color tag bit associated with the data, and evicting thestored data based on the color tag bit.

Another aspect of the present disclosure relates to a device including aa processor, a first sub-cache, and a second sub-cache, coupled inparallel with the first cache, for storing cache data evicted from thefirst sub-cache and write-memory commands that are not cached in thefirst sub-cache, and wherein the second sub-cache includes: color tagbits configured to store an indication that a corresponding cache lineof the second sub-cache storing write-miss data is associated with acolor tag, and an eviction controller configured to evict the cache lineof the second sub-cache storing write-miss data based on the color tagassociated with the cache line.

Another aspect of the present disclosure relates to techniques forcaching data by a caching system, the caching system including a firstsub-cache, and a second sub-cache, coupled in parallel with the firstsub-cache, for storing cache data evicted from the first sub-cache andwrite-memory commands that are not cached in the first sub-cache, thesecond sub-cache including color tag bits configured to store anindication that a corresponding line of the second sub-cache isassociated with a color tag, and an eviction controller configured toevict cache lines of the second sub-cache storing write-memory databased on the color tag associated with the line, and wherein the secondsub-cache is further configured to: receive a first write memory commandfor a memory address, the write memory command associated with a firstcolor tag, store first data associated with the first write memorycommand in a cache line of the second sub-cache, store the first colortag in the second sub-cache, receive a second write memory command forthe cache line, the write memory command associated with a second colortag, merge the second color tag with the first color tag, store themerged color tag, and evict the cache line based on the merged colortag.

Another aspect of the present disclosure relates to a method for cachingdata, including receiving, by a caching system, a write memory commandfor a memory address, the write memory command associated with a firstcolor tag, determining, by a first sub-cache of the caching system, thatthe memory address is not cached in the first sub-cache, determining, bysecond sub-cache of the caching system, that the memory address is notcached in the second sub-cache, wherein the second sub-cache isconfigured to store, in parallel with the first sub-cache, cache dataevicted from the first sub-cache and write-memory commands that are notcached in the first sub-cache, storing first data associated with thefirst write memory command in a cache line of the second sub-cache,storing the first color tag in the second sub-cache, receiving a secondwrite memory command for the cache line, the write memory commandassociated with a second color tag, merging the second color tag withthe first color tag, storing the merged color tag, and evicting thecache line based on the merged color tag.

Another aspect of the present disclosure relates to a device including aprocessor, a first sub-cache, and a second sub-cache, coupled inparallel with the first sub-cache, for storing cache data evicted fromthe first sub-cache and write-memory commands that are not cached in thefirst sub-cache, and the second sub-cache including color tag bitsconfigured to store an indication that a corresponding line of thesecond sub-cache is associated with a color tag, and an evictioncontroller configured to evict cache lines of the second sub-cachestoring write-memory data based on the color tag associated with theline, and wherein the second sub-cache is further configured to receivea first write memory command for a memory address, the write memorycommand associated with a first color tag, store first data associatedwith the first write memory command in a cache line of the secondsub-cache, store the first color tag in the second sub-cache, receive asecond write memory command for the cache line, the write memory commandassociated with a second color tag, merge the second color tag with thefirst color tag, store the merged color tag; and evict the cache linebased on the merged color tag.

Another aspect of the present disclosure relates to a caching systemincluding a first sub-cache, a second sub-cache coupled in parallel withthe first sub-cache, for storing cache data evicted from the firstsub-cache and write-memory commands that are not cached in the firstsub-cache, the second sub-cache including privilege bits configured tostore an indication that a corresponding cache line of the secondsub-cache is associated with a level of privilege, and wherein thesecond sub-cache is further configured to receive a first write memorycommand for a memory address, the first write memory command associatedwith a first level of privilege, store, in a cache line of the secondsub-cache, first data associated with the first write memory command,store, in the second sub-cache, the level of privilege associated withthe cache line, receive a second write memory command for the cacheline, the second write memory command associated with a second level ofprivilege, merge the first level of privilege with the second level ofprivilege, store the merged privilege level, and output the mergedprivilege level with the cache line.

Another aspect of the present disclosure relates to a method for cachingdata, including receiving, by a caching system, a first write memorycommand for a memory address, the first write memory command associatedwith a first privilege level, determining, by a first sub-cache of thecaching system, that the memory address is not cached in the firstsub-cache, determining, by second sub-cache of the caching system, thatthe memory address is not cached in the second sub-cache, wherein thesecond sub-cache is configured to store, in parallel with the firstsub-cache, cache data evicted from the first sub-cache and write-memorycommands that are not cached in the first sub-cache, storing first dataassociated with the first write memory command in a cache line of thesecond sub-cache, storing the first privilege level in the secondsub-cache, receiving a second write memory command for the cache line,the second write memory command associated with a second level ofprivilege, merging the first level of privilege with the second level ofprivilege, storing the merged privilege level, and outputting the mergedprivilege level with the cache line.

Another aspect of the present disclosure relates to a device including aprocessor, a first sub-cache, and a second sub-cache, coupled inparallel with the first sub-cache, for storing cache data evicted fromthe first sub-cache and write-memory commands that are not cached in thefirst sub-cache, the second sub-cache including privilege bitsconfigured to store an indication that a corresponding cache line of thesecond sub-cache is associated with a level of privilege, and whereinthe second sub-cache is further configured to receive a first writememory command for a memory address, the first write memory commandassociated with a first level of privilege, store, in a cache line ofthe second sub-cache, first data associated with the first write memorycommand, store, in the second sub-cache, the level of privilegeassociated with the cache line, receive a second write memory commandfor the cache line, the second write memory command associated with asecond level of privilege, merge the first level of privilege with thesecond level of privilege, store the merged privilege level, and outputthe merged privilege level with the cache line.

Another aspect of the present disclosure relates to a caching systemincluding a first sub-cache, and a second sub-cache coupled in parallelwith the first sub-cache; wherein the second sub-cache includes linetype bits configured to store an indication that a corresponding line ofthe second sub-cache is configured to store write-miss data.

Another aspect of the present disclosure relates to a method for cachingdata including receiving, by a caching system, a write memory requestfor a memory address, determining, by a first sub-cache of the cachingsystem, that the memory address is not cached in the first sub-cache,determining, by second sub-cache of the caching system, the secondsub-cache coupled in parallel with the first sub-cache, that the memoryaddress is not cached in the second sub-cache, storing data associatedwith the write memory request in the second sub-cache, and storing, in aline type bit of the second sub-cache, an indication that the storeddata corresponds to a write-miss.

Another aspect of the present disclosure relates to a device including aprocessor, a first sub-cache, and a second sub-cache coupled in parallelwith the first sub-cache; wherein the second sub-cache includes linetype bits configured to store an indication that a corresponding line ofthe second sub-cache is configured to store write-miss data.

Another aspect of the present disclosure relates to a caching systemincluding a first sub-cache, a second sub-cache, coupled in parallelwith the first sub-cache, for storing cache data evicted from the firstsub-cache and write-memory commands that are not cached in the firstsub-cache, and a cache controller configured to receive two or morecache commands, determine a conflict exists between the received two ormore cache commands, determine a conflict resolution between thereceived two or more cache commands, and sending the two or more cachecommands to the first sub-cache and the second sub-cache.

Another aspect of the present disclosure relates to a method for cachingdata including receiving two or more cache commands, determining aconflict exists between the two or more cache commands, determining aconflict resolution between the received two or more cache commands, andsending the two or more cache commands to a first sub-cache and a secondsub-cache, wherein the second sub-cache is configured to store, inparallel with the first sub-cache, cache data evicted from the firstsub-cache and write-memory commands that are not cached in the firstsub-cache.

Another aspect of the present disclosure relates to a device including aa processor, a first sub-cache, and a second sub-cache, coupled inparallel with the first sub-cache, for storing cache data evicted fromthe first sub-cache and write-memory commands that are not cached in thefirst sub-cache, and a cache controller configured to receive two ormore cache commands, determine a conflict exists between the receivedtwo or more cache commands, determine a conflict resolution between thereceived two or more cache commands, and sending the two or more cachecommands to the first sub-cache and the second sub-cache.

BRIEF DESCRIPTION OF THE DRAWINGS

For a detailed description of various examples, reference will now bemade to the accompanying drawings in which:

FIG. 1 is a block diagram of a computer system.

FIG. 2 is a block diagram illustrating a simplified cache memorypipeline for processing a read request.

FIG. 3 is a block diagram of a cache memory pipeline with a store queue,in accordance with aspects of the present disclosure.

FIG. 4 is a block diagram illustrating a cache architecture including avictim cache.

FIG. 5 is a block diagram illustrating a cache architecture including aparallel victim cache, in accordance with aspects of the presentdisclosure.

FIG. 6 is a block diagram illustrating a victim cache memory pipeline,in accordance with aspects of the present disclosure.

FIG. 7 is a block diagram illustrating a victim cache pipeline includinga write-miss buffer, in accordance with aspects of the presentdisclosure.

FIG. 8 is a block diagram of a MESI RAM, in accordance with aspects ofthe present disclosure.

FIG. 9 is an architectural block diagram of a cache system including awrite-miss buffer, in accordance with aspects of the present disclosure.

FIGS. 10-16 are flow diagrams illustrating techniques for caching data,in accordance with aspects of the present disclosure.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of a computer system 100. The computer system100 includes a data cache 102, such as a level one (L1) data cache. Thedata cache 102 stores a subset of the system's data to reduce the timeneeded to access (e.g., read and/or write) the cached subset. Byeffectively caching the most commonly used data, the data cache 102 maymarkedly improve system performance.

The data cache 102 may be coupled to one or more processing resources104 (e.g., processor cores) and to an extended memory 106. The extendedmemory 106 includes other levels of a memory hierarchy, such as an L2cache, storage devices, etc. The data cache 102 may be incorporated intothe same die as the processing resource(s) 104 (e.g., on-die cache) ormay be on a separate die. In either case, the cache 102 is coupled toeach processing resource 104 by one or more interfaces used to exchangedata between the cache 102 and the processing resource. In this example,the cache 102 is coupled to each processing resource 104 by a scalarinterface and a vector interface. In examples with more than oneinterface, in the event that one interface is busy, a command may beserviced using another interface. For example, when a scalar readcommand is received by the cache over the scalar interface, theassociated data may be provided to the processing resource 104 over thevector interface based on interface utilization, data size, and/or otherconsiderations. Similarly, the cache 102 may also be coupled to theextended memory 106 by one or more interfaces. Where more than oneinterface is present, an interface may be selected based on utilization,data size, and/or other considerations.

Each interface may have any suitable width. The widths of the interfacesmay be different from each other, although in many examples, they areinteger multiplies of the narrowest interface. In one such example, thescalar interface is 64 bits wide, the vector interface is 512 bits wide,and the extended memory interface is 1024 bits wide.

The interfaces may be bidirectional or unidirectional. Bidirectionalinterfaces may include two independent unidirectional interfaces so thatdata can be transmitted and received concurrently. In one such example,the vector interface includes two 512-bit unidirectional busses, one forreceiving data and operations from the processing resource 104, and onefor sending data to the processing resource 104.

The data cache 102 may include a number of pipelines for processingoperations received via these interfaces. FIG. 2 is a block diagramillustrating a simplified cache memory pipeline 200 for processing aread request. As shown in the cache memory pipeline 200, a processor 202sends a memory request to a cache memory 204. While the cache memory 204is described in the context of a L1 cache, the concepts discussed hereinmay be understood as applicable to any type of cache memory. In certaincases, the memory request may be sent via a cache or memory controller,not shown. In this example, the cache memory pipeline includes fivestages, E1, E2, E3, E4, and E5. Each cache memory pipeline stage may beallocated a specific number of clock cycles to complete, and in someexamples, each stage is allocated one clock cycle such that cached datamay be returned to the processor 202 after the E5 memory pipeline stagein five clock cycles. In the E1 memory pipeline stage, a memory requestis received by the cache memory 204. The memory request includes amemory address of the data to be retrieved from. In the E2 pipe stage, atag random access memory (RAM) 206 is read to determine what memoryaddresses are currently store in the cache memory. The tag RAM 206stores a table that records which entries in the memory 208 correspondto which memory addresses in an extended memory. The tag RAM may be abank or portion of memory used to hold the table of memory addresses. Incertain cases, the cache may be a N-way associative cache where eachcache set can hold N lines of memory addresses. As N increases, thenumber of addresses which are searched through also increases, which inturn increases the amount of time needed to determine whether arequested memory address is in the tag RAM. In the E3 memory pipelinestage, the received memory address is compared to memory addressesstored read from the tag RAM to determine whether there is a cache hitor miss. A cache hit occurs when data associated with a requested memoryaddress is stored in the cache and a cache miss occurs when dataassociated with the requested memory address is not stored in the cache.In the E4 memory pipeline stage, a portion of the memory 208 associatedwith the requested memory address is read and at the E5 memory pipelinestage, the requested memory address is provided to the processor 202.The memory 208 may be any type of memory suitable for a cache memory,such as dynamic random-access memory (DRAM), static random-access memory(SRAM), registers, etc. It may be understood that the pipeline stages,as described, are for illustrating how a memory pipeline can beconfigured, and as such, omit certain sub-steps and features. In certainimplementations, the stages in which certain activities, such as thememory access, are performed may differ.

FIG. 3 is a block diagram illustrating a cache pipeline 300 supportingread-modify-write with error-correcting code store queue architecture,in accordance with aspects of the present disclosure. The pipeline 300includes a read path (with read path latches 302, tag RAM 306, memory308, etc.), and write path (write path latches 304, store queue 314,etc.).

With respect to the read path, the pipeline 300 includes a tag RAM 306and a memory 308 (e.g., DRAM or other suitable memory). The cache mayhave any degree of associativity, and in an example, the cache is adirect-mapped cache such that each extended memory address correspondsto exactly one entry in the cache memory 308.

In certain cases, the cache pipeline 300 may include support for ECC andthe memory 308 may be coupled to an error detection and correctioncircuit 310. In an ECC example, the memory 308 stores data in blocksalong with a set of ECC syndrome bits that correspond to the blocks.When a read operation is received, the memory 308 may provide the storeddata block and the corresponding ECC syndrome bits to the errordetection and correction circuit 310. The error detection and correctioncircuit 310 may regenerate the ECC syndrome bits based on the data blockas read from the memory 208 and compare the regenerated ECC syndromebits with those that were previously stored. Any discrepancy mayindicate that the data block has been read incorrectly, and the ECCsyndrome bits may be used to correct the error in the data block. Theability to detect and correct errors makes the cache well-suited tomission critical applications.

An arbitration unit 312 may be coupled to the memory 308 to arbitratebetween conflicting accesses of the memory 308. When multiple operationsattempt to access the memory 308 in the same cycle, the arbitration unit312 may select which operation(s) are permitted to access the memory 308according to a priority scheme. Many different priority schemes may beused. As an example of a priority scheme, the arbitration prioritizesread operations over write operations because write data that is in thepipeline 300 is available for use by subsequent operations even beforeit is written to the memory 308, for example via a data forwardingmultiplexer 318 of a store queue 314, as will be discussed in moredetail below. Thus, there is minimal performance impact in allowing thewrite data to wait in the pipeline 300. However, as the pipeline 300fills with write data that has not yet been written back, the priorityof the write operations may increase until they are prioritized overcompeting read operations.

The read path may run in parallel with the store queue 314. Because aread operation may refer to data in a write operation that may not havecompleted yet, the pipeline 300 may include write forwardingfunctionality that allows the read path to obtain data from the storequeue 314 that has not yet been written back to the memory 308. In anexample, the pipeline 300 includes a pending store address table 316that records the addresses of the operations at each stage of the storequeue 314, the data forwarding multiplexer 318 to select data from oneof the stages of the store queue 314 for forwarding, and a store queuehit multiplexer 320 that selects between the output of the memory 308(by way of the error detection and correction circuit 310) and theforwarded store queue 314 data from the data forwarding multiplexer 318.

An example flow of a read operation through the pipeline 300 will now bedescribed. In a first cycle, indicated by stage E1, the cache retrievesa record from the tag RAM 306 that is associated with an address of theread operation to determine whether the data is stored in the cache'smemory 308. In a direct mapped example, the cache does not need to waitfor the tag RAM comparison before requesting data from the memory 308,and thus, the tag RAM comparison between the address of the readoperation and the record of cached addresses does not need to extendinto a second (E2) or third (E3) cycle.

In the second cycle, in stage E2, the cache may request the data and ECCsyndrome bits from the memory 308, if the arbitration unit 312 permits.In this cycle, the cache may also determine whether newer data isavailable in the store queue 314 by comparing the read address to thepending store address table 316. If so, the data forwarding multiplexer318 is set to forward the appropriate data from the store queue 314.

Data and ECC syndrome bits may be provided by the memory 308 in thethird cycle in stage E3. However, this data may or may not correspond tothe memory address specified by the read operation because the cache mayallocate multiple extended memory addresses to the same entry in thecache's memory 308. Accordingly, in the third cycle, the cachedetermines whether the provided data and ECC from the memory 308corresponds to the memory address in the read operation (e.g., a cachehit) based on the comparison of the tag RAM record. In the event of acache hit, the data and ECC bits are received by the error detection andcorrection circuit 310, which corrects any errors in the data in afourth cycle in stage E4.

As explained above, newer data that has not yet been written to thememory 308 may be present in the store queue 314, and may be forwardedfrom the store queue 314 by the data forwarding multiplexer 318. If so,the store queue hit multiplexer 320 selects the forwarded data over thecorrected data from the memory 308.

Either the corrected data from the memory 308 or the forwarded data fromthe store queue 314 is provided to the requesting processor in a fifthcycle in stage E5. In this way, the example cache may provide data tothe processor with full ECC checking and correction in the event of acache hit in about 5 cycles.

In the event that the data and ECC bits are not present in the memory308 (e.g., a cache miss), the pipeline 300 may stall until the data canbe retrieved from the extended memory, at which point the data may bewritten to the memory 308 and the tag RAM 306 may be updated so thatsubsequent reads of the data hit in the cache.

The cache may also support a number of operations that read data fromthe cache and make changes to the data before rewriting it. For example,the cache may support read-modify-write (RMW) operations. A RMWoperation reads existing data, modifies at least a portion of the dataand overwrites that portion of the data. In ECC embodiments, a RMWoperation may be performed when writing less than a full bank width. Asthe write is not for the full bank width, performing an ECC operation onjust the portion of the data being written would result in an incorrectECC syndrome. Thus, the read functionality of the RWM is used becausethe portion of the data in the bank that will not be overwritten stillcontributes to the ECC syndrome bits.

A RMW operation may be split into a write operation and a readoperation, and the pipeline 300 may be structured such that the readoperation in the read path stays synchronized with the write operationin the store queue 314. The read operation and the write operationremain synchronized until a read-modify-write merge circuit 322overwrites at least a portion of the read data with the write data toproduce merged data. The merged data is provided to an ECC generationcircuit 324 that generates new ECC syndrome bits for the merged data,and then the merged data and ECC syndrome bits may be provided to thearbitration unit 312 for storing in the memory 308.

An example flow of a RMW operation through the pipeline 300 will now bedescribed. The read portion of the operation proceeds substantially asexplained above in stages E1-E4, where the cache compares an address ofthe read operation to a record of the tag RAM 306, and the cacherequests the data and ECC syndrome bits from the memory 308 and/or thestore queue 314. Because the RMW operation will modify the data, inexamples that track MESI (Modified, Exclusive, Shared, and Invalid)states of entries in the memory 308, a cache hit that is not in eitherthe Modified or Exclusive state may be considered a cache miss. When thedata is obtained in the proper state and any errors are corrected, it isprovided to the read-modify-write merge circuit 322 in cycle E4 (orlater in the event of a cache miss). In this same cycle, theread-modify-write merge circuit 322 may overwrite at least a portion ofthe corrected data with the write data to produce merged data. The ECCgeneration circuit 324 generates new ECC syndrome bits for the mergeddata in stage E5 (or later in the event of a cache miss). The mergeddata and the ECC syndrome bits are provided to the arbitration unit 312for writing to the cache memory 308.

In some examples, sequential RMW operations are received that refer tothe same address. Rather than wait for the merged data from the earlierRMW operations to be written to the memory 308, the store queue 314 mayinclude address comparators 326 for write forwarding that may feed themerged data back to a prior stage of the store queue 314 for use by asubsequent RMW operation. This may be referred to as “piggy-backing.”The data may be fed back before or after the ECC generation circuit 324.As the feedback effectively merges the RMW operations, the final RMWoperation has a complete set of data and ECC syndrome bits. Accordingly,earlier-in-time RMW operations may be canceled before they are writtenback to the memory 308. This may avoid stalling other operations withwrites of obsolete data.

In certain cases, memory 308 may represent the entirety of the datacache. By way of example only, in such an embodiment, the data cache(which may be an L1 data cache) is associated with a single store queuestructure 314. As an example, the data cache may include 256 rows, witheach row having 1024 bits (1 Kb) per row.

In other examples, the cache may be divided into a plurality ofindependently addressable banks, and each individual bank may have itsown respective store queue structure 314. For instance, consider anembodiment in which the above-mentioned data cache has 256 rows witheach row having a line width of 1024 bits, but being divided into 16banks, with 64 bits per row in a given bank. In such an embodiment,there would be 16 store queues, one for each bank of the data cache.Thus, read and write operations may be sent to the banks in parallel,and each bank arbitrates its own processes in response to the readand/or write operations. By allowing each bank of a multi-bank cache tooperate independently, operation of the cache is more efficient since anentire cache line is not locked up when a request is received. Rather,only the portion of the cache line allocated to the bank that receivedsuch a request would be locked. Of course, the cache size describedabove is only one example and the disclosure is not limited to anyparticular cache line width, number of banks, or rows, etc.

The above example is also useful for writing and/or reading vector data.For instance, vector data may be 512 bits wide. For a multi-bank cache,a write request containing vector data that is a hit in the cache may beprocessed as 8 parallel writes to 8 banks (e.g., 8×64 bits=512 bits).Similarly, a read request to such a multi-bank cache could be performedas 8 parallel reads from 8 banks.

Another feature that may be present in a contemplated embodiment of thecache system is supporting inflight forwarding and invalidation. Forinstance, assume in one example that the cache is a two-way setassociative cache. In a two-way set associative implementation, eachcache line within the cache could map to two different addresses in ahigher level of memory (e.g., L2 cache or main system memory forinstance).

Consider a situation in which a given cache line, referred to as “Line1” in this example, is a cache line in a two-way set associate cache andmaps to two different addresses in memory, which are referred to as“Address A” and “Address B.” Now, suppose that cache receives a firstrequest is received which is a partial write (e.g., a write to less thana full cache line) and is followed by a read request, like so:

Write (Address A): Data 1 //write Data1 to cache line corresponding toAddress A

Read (Address B) //read data contained in cache line corresponding to//Address B

In this example, let us assume that the Write request is a hit, meaningthat the cache line corresponding to Address A, which we will assume isLine 1 in this example, is in the cache. In response, the cache systemwill begin the process of writing Data 1 to Line 1.

Subsequently, let us assume that the Read request is a miss. In otherwords, while Line 1 may be mapped to either Address A or Address B inthis two-way set associative cache, in this instance it is presentlymapped to Address A. The miss thus occurs because Address B is not foundin the tag RAM 306. In response to the miss, the cache system willservice the miss by going to retrieve the data from a higher level ofmemory, such as an L2 cache and, when the data, which is referred to as“Data 2” in this example is returned, data currently in Line 1 will beevicted in order to store Data 2. This may occur before the earlierWrite has completed. In this case, there is no need to complete theWrite operation since the Line 1 is going to be evicted anyways. Thus,the Write can be invalidated prior to its completion, referred to as“inflight.”

Atomic operations, such as an atomic compare and swap, are furtherexamples of memory operations that include a read. An atomic compare andswap operation may provide an address, a key value, and a set of newdata to be written. The current data at the address is compared to thekey, and if they match, the current data at the address is overwrittenby the new data. Accordingly, in some examples, the pipeline 300includes an atomic compare and swap circuit 328 to perform thecomparison and forward the new data on for writing if the current datamatches the key value. The atomic compare and swap circuit 328 may becoupled in parallel with the read-modify-write merge circuit 322 and afunction multiplexer 330 may be used to determine which output isprovided to the ECC generation circuit 324.

An example flow of an atomic compare and swap operation through thepipeline 300 will now be described. The read portion of the operationproceeds substantially as explained above in stages E1-E4, where thecache compares an address of the read operation to a record of the tagRAM 306 and the cache requests the data and ECC syndrome bits from thememory 308 and/or the store queue 314. Because the atomic compare andswap operation may modify the data, in examples that track MESI statesof entries in the memory 308, a cache hit that is not in either theModified or Exclusive state may be considered a cache miss. When thedata is obtained in the proper state, it is provided to the atomiccompare and swap circuit 328 in stage E4 (or later in the event of acache miss). During these cycles, the new data and the key may be storedin the write path latches 304. During stage E4, the atomic compare andswap circuit 328 may receive the corrected read data, the key, and thenew data. If the corrected read data matches the key, the atomic compareand swap circuit 328 provides the new data to the ECC generation circuit324. The atomic compare and swap circuit 328 may also change the stateof the new data to Modified if the read data matches the key. The ECCgeneration circuit 324 generates new ECC syndrome bits for the new datain stage E5 (or later in the event of a cache miss). The new data andthe ECC syndrome bits are provided to the arbitration unit 312 forwriting to the cache memory 308.

The pipeline 300 may also perform write forwarding using the addresscomparators 326 when sequential atomic compare and swap operations arereceived, similar to that described in the context of the RMWoperations.

Further examples of memory operations that read existing data includehistogram operations. Histogram operations may specify a value to add toor subtract from existing data. To support histogram operations, thepipeline 300 may include an arithmetic unit 332 coupled in parallel withthe read-modify-write merge circuit 322 and/or the atomic compare andswap circuit 328. The arithmetic unit 332 may support any size operands,and in one such example, the arithmetic unit 332 supports addition andsubtraction of a 16-bit value from a 64-bit bank. Multiplexer 330 may beused to select between the arithmetic unit 332, the read-modify-writemerge circuit 322, and the atomic compare and swap circuit 328.

An example flow of a histogram operation through the pipeline 300 willnow be described. The read portion of the operation proceedssubstantially as explained above in stages E1-E4, where the cachecompares an address of the read operation to a record of the tag RAM 306and the cache requests the data and ECC syndrome bits from the memory308 and/or the store queue 314. Because the histogram operation willmodify the data, in examples that track MESI states of entries in thememory 308, a cache hit that is not in either the Modified or Exclusivestate may be considered a cache miss. When the data is obtained in theproper state, it is provided to the arithmetic unit 332 in stage E4 (orlater in the event of a cache miss). During these cycles, the value tobe added to the corrected data may be stored in the write path latches304. During stage E4, the arithmetic unit 332 may receive the correctedread data and the value to be added or subtracted from the read data.The arithmetic unit 332 performs the specified addition or subtractionand provides the new data to the ECC generation circuit 324. The ECCgeneration circuit 324 generates new ECC syndrome bits for the new datain stage E5 (or later in the event of a cache miss). The new data andthe ECC syndrome bits are provided to the arbitration unit 312 forwriting to the cache memory 308.

The pipeline 300 may also perform write forwarding using the addresscomparators 326 when sequential histogram operations are received,similar to that described in the context of the RMW operations.

While FIG. 3 illustrates a single pipeline 300 with a single instance ofa store queue 314, latches 302 and 304, an error detection andcorrection circuit 310, etc. However, as described above, the memory 308may be arranged to support more than one independent copy of thepipeline 300. Accordingly, the pipeline 300 may be reproduced multipletimes. In one such example, the memory 308 entries are 1024-bit blocksbased on the width of the extended memory interface, and the memory 308is arranged into 16 parallel banks where each 1024-bit block includes 64bits in each of the 16 banks. In this particular example, each of the 16banks has its own independent pipeline 300. A read or write operationfrom the processor is divided into a number of bank-level operations andeach is provided to its respective pipeline 300 so that they may beprocessed concurrently. For example, a 512-bit vector read operation maybe divided into eight 64-bit read operations that are provided to eightindependent pipelines 300 to be processed in parallel. At the same time,another operation may be received over the scalar interface or a vectorinterface of another processor and divided into operations that areprovided to the remaining pipelines 300. Accordingly, the data cache mayservice multiple independent operations concurrently, with eachoperation being allocated a subset of the pipelines 300.

In certain cases, the cache pipeline 300 may be coupled to a victimcache. Victim caches may help improve cache miss rates, such as thosedue to conflict misses. A victim cache, in certain cases, can have animpact on cache performance similar to increasing the N-way associativenumber (e.g., a degree of cache associativity). This benefit may be moreevident in cases where the victim cache is added to a direct-mappedcache, as a direct mapped cache typically has the highest rate ofconflict misses. The addition of a victim cache may help improve anoverall cache hit rate.

FIG. 4 is a block diagram illustrating a cache architecture 400including a victim cache. Here, the cache architecture 400 includes amain cache 404 coupled to a victim cache 402, where the victim cache 402resides after the main cache 404. When a line of data is evicted out ofthe main cache 404, rather than moving the line of data to a next levelcache (e.g., L2), the line of data is moved to the victim cache 402 andheld there until there is a reason to evict it out, such as due tocapacity or other reasons. In such a cache architecture 400, in case ofa cache miss from the main cache 404, the victim cache 402 can servicethe miss and return the line to the processor 414, reducing the overalllatency experienced for a read miss.

As an example, suppose a processor 414 attempts to access address A inthe main cache 404 and the victim cache 402, causing a cache miss. Themain cache 404 may then evict address B from the main cache 404 andcreate a victim cache (Address B) line in the victim cache 402. The readmiss of address A then goes out 406 to a higher level memory 412, suchas L2 cache, another cache, and/or other memory, to be serviced. Onceaddress A is accessed, the contents of address A may be returned 410 andstored in the main cache 404.

If a subsequent access attempt for address B is issued, the main cache404 is checked to see if address B is cached, resulting in a cache miss.The victim cache 402 is then checked 408, which returns a hit as addressB was stored in the victim cache 402 after being evicted from the maincache 404. Address B may then be restored 410 to the main cache 404 andreturned as a result to the processor 414. The read latency in such asystem, while faster as compared to having to fetch the address fromhigher level memory 414 (e.g., L2 cache, main memory, etc.), is stillgreater than the latency when there is a cache hit in the main cache404.

To help address the latency experienced in cache architectures whichplace the victim cache after the main cache, in certain cases, thevictim cache may be placed in parallel with the main cache. FIG. 5 is ablock diagram illustrating a cache architecture 500 including a parallelvictim cache, in accordance with aspects of the present disclosure. Thecache architecture 500 can be thought of as having multiple sub-cachesin parallel. Each of the main cache 504 and the victim cache 502 hastheir own independent associativity and the overall cache system 500 mayhave the appearance of a hybrid way cache system. In certain cases,either the main cache 504 or the victim cache 502 may be n-wayassociative, while the other cache may be fully associative. In thisexample, the main cache 504 may be a direct mapped (1-way associative)cache and the victim cache 502 may be fully associative.

In this example, if a processor 514 issues a read to address A, whichmisses in the main cache 504, the miss request goes out to a higherlevel cache 512 for servicing and the victim line, here address B, ismoved to the victim cache 502. Next, let us assume the processor 514wants to read address B. In this example, the address of the read ischecked against both the main cache 504 and the victim cache 502 inparallel. This results in a read miss in the main cache 504 but a hit inthe victim cache 502. The read results for address B may then bedirectly returned from the victim cache 402 to the processor 514. Asboth caches are checked in parallel, the read latency remains the sameas if the read request was a hit in the main cache 504. The overalleffect of having a victim cache in parallel to the main cache is that weget the benefit of reducing the cache miss rate while also not impactingthe latency in case the subsequent CPU access hits the victim cache.

FIG. 6 is a block diagram illustrating a victim cache memory pipeline600, in accordance with aspects of the present disclosure. As discussedabove, the victim cache may be a fully associative cache, meaning thatthere is not a mapping addresses and cache lines. Rather any address maybe stored in any cache line of the victim cache. Thus, for an addressreceived at block 602 in the E1 stage, the received address is checked,in the E2 stage, against addresses stored in each location in the victimcache tag RAM 604. In certain cases, these locations in the victim cachetag RAM 604 correspond to lines of the victim cache 606. For example, ifaddress A is received at block 602, address A is compared againstaddresses in the victim cache tag RAM 604 to see if there is a match.If, for example, location 5 of the victim cache tag RAM 604 correspondsto address A, then the data corresponding to address A is stored inaddress 5 of the victim cache 606. This data is eventually returned tothe processor at block 608.

Where the victim cache operates in parallel to the main cache, it may bedesirable for the victim cache to support certain features of the maincache, such as ECC and RMW operations. In certain cases, the victimcache may also include a store queue 610 structure to help providesupport for RMW and ECC operations, in a manner similar to the storequeue 314 as discussed in conjunction with FIG. 3.

In certain cases, caches may include a write-miss buffer. A write-misscan occur when a memory write is issued to a cache against a memoryaddress that is not in the cache. The memory write may be an atomicoperation, the processor that issued the memory write may stall until amemory write complete message is returned to the processor. Rather thanstalling the processor until a write to a higher level memory iscomplete, the cache may store the memory write in a write-miss bufferand returns an indication that the memory write is complete. Thewrite-miss buffer this information and then outputs (e.g., drains) thememory write to the target memory address. Once the memory write outputto the target memory address is complete, the memory write is removedfrom the write-miss buffer. To output the memory write, the write-missbuffer may include three pieces of information for each memory write:the memory address for the write, the corresponding data, and the byteenable (byten) mask for the memory write. Generally, the byte enablelogic or mask that defines a portion of the memory line to be writtento. Typically, write-miss buffers are implemented using dedicated buffercircuitry as a first in first out (FIFO) buffer. However, FIFOwrite-miss buffers are typically configured to drain to a higher levelmemory more or less continuously and do not support features such aswrite merging. In certain cases, a write-miss buffer may be allocatedinto the cache system using the victim cache.

FIG. 7 is a block diagram illustrating a victim cache pipeline 700including a write-miss buffer, in accordance with aspects of the presentdisclosure. As discussed above, a write-miss buffer may store the memoryaddress for the write, the data to be written, and the byten logic. Inthis example victim cache pipeline 700 the memory addresses for awrite-miss buffer entry may be stored in a victim cache tag RAM 704 andthe corresponding data may be stored in the victim cache RAM 706. A byteenable memory 702 may also be added to the victim cache to store thebyten logic associated with a write-miss buffer line. In certain cases,the MESI RAM 732 may be modified to include an extra bit indicating aline type 708 associated with an entry in the victim cache RAM 706. Theline type 708 indicates whether a particular line of the victim cacheRAM 706 is a cache line or a write-miss buffer line. In certain cases,the line type 708 may be added to another memory, such as the victimcache tag RAM 704, rather than the MESI RAM 718.

Additionally, the victim cache system, for example as discussed inconjunction with FIG. 6, may be configured as a write-miss buffer byadding control logic in read and write-miss state machines to addressallocating of write-miss entries inside the victim cache, as well aslogic to support handling various address hazards in cases like a readto the same line as a write-miss entry inside the victim cache(selective write-miss flush), and logic to support resolving sameaddress load/stores in data-paths of the victim cache. A storage queue710 structure may also be provided to enable support for RMW and ECCsupport for write-misses. In certain cases, the storage queue 710 may beused to enable support for RMW and ECC support for write miss operationsas well as cache operations.

As an example of performing write-miss buffer operations in a cache,when a write is received at block 712, the address of the write ischecked against memory addresses stored in each location in the victimcache tag RAM 704. A write-miss occurs when the address of the write isnot found in the victim cache tag RAM 704. When operating as awrite-miss cache, a determination is made where to store the write-missin the victim cache. In certain cases, this determination may be madebased on entries in the victim cache tag RAM 704. If there is an emptylocation in the victim cache tag RAM 704, then the address of thewrite-miss is stored in the victim cache tag RAM 704 in the emptylocation. If there is not an empty location, then a line may be evictedbased on an eviction scheme for allocating the write-miss buffer space.In other cases, the line may have previously been used as a write missbuffer or a cache line. The line type 708 corresponding to the locationin the victim cache tag RAM 704 of the write-miss may be updated toindicate that the location is being used as a write-miss buffer. Forexample, a 0 in the line type 708 may indicate the corresponding line isa cache entry, while a 1 in the line type 708 may indicate that thecorresponding line is a write-miss buffer entry. In certain cases, theline type 708 may be store in another memory location, such as the MESIRAM 732, or in a dedicated memory space, such as a set of registers. Thedata of the write-miss is stored in a line of the victim cache RAM 706corresponding to the victim cache tag RAM 704 location used. The byteenable logic may be stored in a corresponding slot of the byte enablememory 702. In certain cases, the byte enable memory 702 may includebits indicating which positions of the byte enable memory 702 are valid.The line type bit helps enable storing write miss entries in lines ofthe victim cache RAM 706 without having to dedicate a portion of thevictim cache RAM 706 as a write buffer.

According to aspects of the present disclosure, the victim cacheeviction logic, for example in an eviction controller 714, may beupdated to handle write-miss entries stored in the victim cache toprioritize evicting write-miss entries in favor of store cache entries.Additionally, control logic for read and write state machines of thevictim cache controller 734 may be configured to determine that anincoming read request might hit a cache line of the victim cacheconfigured to store a write-miss. In such cases, the victim cachecontroller 734 may be configured to treat this read hit like a read-missand evict the stored write miss command to higher level memory and thensend the read miss out to the higher level memory. Additionally, cachesystem are typically configured to stall on a miss while a write-missbuffer typically does not stall a processor as it buffers the writemiss. To help address this difference, the victim cache controller 734may be configured to allocate write miss entries to the victim cachewithout stalling the CPU by speculatively reading an incoming write dataso that data can be stored directly in to the victim cache withoutstalling if it is a write miss.

In certain cases, by placing write-misses in a cache system, thewrite-misses may stay cached until there is a requirement to evict themout to higher level memories. In certain cases, a write-miss stored inthe victim cache may stay in the cache system until a defined conditionis met, a fence operation is requested, or there are address/hardwareconflicts which need a particular write-miss entry to be evicted.

Maintaining the write-miss buffer until an explicit fence operation isrequested may be problematic, for example, if a fence operation is notraised at the end of an operation, and any write-misses stored in thevictim cache may be stuck inside the victim cache. Additionally, therecan be latency concerns from fence operations, as flushing out (e.g.,evicting all) write miss buffers stores only in response of a fenceoperation can result in delays as the cache needs to drain the storedwrite-misses out to the appropriate memories and then wait for theacknowledgments to return. In certain cases, an auto-flushing ordraining of write-miss entries stored in the victim cache may beprovided. Auto-flushing may be implemented to help drain write-missentries from the victim cache while reducing the potential latency thatmay be incurred in the drain process. In certain cases, an evictioncontroller 714 may monitor activity on a processor interface. If theeviction controller 714 detects idle activity on the processorinterface, the eviction controller 714 may proceed to start drainingwrite-miss entries from the victim cache. For example, if the evictioncontroller 714 detects that the processor has not issued any commandsfor N number of clock cycles, then the eviction controller 714 mayinternally start an auto-flush process to start draining the storescached inside victim cache. In certain cases, N may be predetermined. Inother cases, N may be configurable. In certain cases, the auto-flushprocess may walk (e.g., examine in order) the VTag RAM looking forwrite-miss entries and flushing entries corresponding to write-missentries. In other cases, the auto-flush process may include logic toflush entries based in a FIFO, least recently used (LRU), random, orother basis.

In auto-flushing write-miss entries, the eviction controller 714 maysend an eviction trigger request to the victim cache controller 734 fora selected write buffer cache line. The victim cache controller 734checks the store queue to see if there are any entries in the storequeue corresponding to a write buffer cache line to be flushed. If thereis a corresponding entry in the store queue, then the victim cachecontroller 734 may wait for the store queue to complete. Once the storequeue is emptied, the victim cache controller 734 may evict the selectedcache lines. The victim cache controller 734 sets up the creation ofstore miss packets to be sent to higher level caches or other memories.These store miss packets may include the corresponding memory addressstored in the victim cache tag RAM 704, byte enable logic in the byteenable memory 702, and the data in the victim cache RAM 706. The victimcache can then send the store miss packet, for example to a cachecontroller of a higher level cache or other memory. When flushingwrite-miss entries, the write miss packet may be sent out to othermemories and the corresponding entries in the victim cache RAM 706,victim cache tag RAM 704, and byte enable memory 702 may be invalidated.The write miss packet may be placed in a drain buffer 716 temporarilyuntil the write miss packet is accepted (e.g., acknowledged) by theother memory. Once the write miss buffer cache line is invalidated, theeviction controller 714 may select a next write miss buffer cache lineto evict and proceed to evict the next write miss buffer cache line. Ifactivity is subsequently detected on the processor interface, thisauto-flush operation may be cancelled. By doing so, the victim cache isable to utilize idle periods and avoid impacting processor performancewhen there is activity on the interface by attempting to drain thewrite-miss buffer entries.

In certain cases, write-misses stored in the victim cache may be evictedbased on a state of the write miss line storing a particular write-miss,rather than a FIFO, LRU, random, or other scheme based on the orderwhich a cache line is allocated. Where the cache is configured to mergedata in the cache, cache writes to a memory line from a processor do notimply that the processor needs to read from other portions of the memoryline. Further, where a processor has written out a large portion of thememory line, such as half the memory line or the whole memory line, theprocessor is less likely to need to access (such as merging additionaldata into) that portion of the memory line again, as compared to wherethe processor has written only a small portion of the memory line. Asthe processor is less likely to need to access a given write miss cacheline after writing out a half or the full write miss cache line, thiswrite miss cache line can be evicted with less potential impact toperformance. Thus, once a write miss cache line is fully written to, orhalf written to, and a write has started on a next entry in the victimcache on another write miss cache line, the full, or half full, cacheline may be evicted. This helps address the case where a processor isissuing a stream of cache writes-misses which are collectively largerthan the victim cache. Additionally, aggressively evicting full or halffull write-miss cache lines helps avoid performance issues that mayarise when a large number of write-miss items stored in the victim cacheare all flushed at once due to a fence operation at the end of anoperation.

To help support removing full or half full write-miss cache lines, thebyte enable memory 702 may include two status bits, a half-full linestatus bit 718 and a full line status bit 720. Each line of the byteenable memory 702 corresponds with a cache line in the victim cache ram206 and each line of the byte enable memory 702 may include thehalf-full line status bit 718 and the full line status bit 720. Thesestatus bits may be updated each write cycle where bits are written intoa cache line configured as a write miss buffer and the status bits maybe calculated based on the bits already stored in the corresponding lineof the victim cache RAM 706, as well as the bits that are being writtento the victim cache RAM 706 in a write cycle. In certain cases, thisupdating may be performed for each cache line of the byte enable memory702 for each cycle and updating may be omitted when there is not a writecycle. As indicated above, each line of the byte enable memory 702 caninclude a bit mask indicating which positions (e.g., bytes) of the byteenable memory 702 are valid (e.g., written to). Determining which linesare full or half full can be performed using an XOR over correspondingportions of the bit mask. The half-full line status bit 718 may be setwhen half of a given cache line has been written to, either the leastsignificant bits, or the most significant bits. Similarly, the full linestatus bit 720 may be set when an entire cache line has been written to.The cache lines may then be walked (e.g., examined in order) todetermine an address to return that has a full or half full cache line.In certain cases, addresses corresponding to full cache lines areselected first and addresses corresponding with half full cache linesare returned if there are no full cache lines. If multiple cache linesare full (or half full, if there are no full cache lines), then thefirst cache line that is full (or half full) found when walking thecache lines may be selected. If no cache line is full or half full, awrite-miss cache line may be selected based on FIFO, LRU, or other logicmay be used to determine which cache line to select where there aremultiple lines available.

Evicting write-miss entries may operate similarly to auto-flush exceptthat the store queue is not checked to see if there are any entriescorresponding to a full or half full write miss cache line being evictedas this operation is initiated by a read or write miss and is performedafter a line has been written to. Once a full or half full write misscache line has been selected for eviction, the eviction controller 714indicates to a victim cache controller 734 to evict the selected cachelines. The victim cache controller 734 sets up the creation of storemiss packets to be sent to higher level caches or other memories. Thesestore miss packets may include the corresponding memory address storedin the victim cache tag RAM 704, byte enable logic in the byte enablememory 702, and the data in the victim cache RAM 706. The store misspackets are sent, for example to a cache controller of a higher levelcache or other memory and the corresponding entries in the victim cacheRAM 706, victim cache tag RAM 704, and byte enable memory 702 may beinvalidated. The store miss packet may be placed in a drain buffer 716temporarily until the store miss packet is accepted (e.g., acknowledged)by the other memory.

Once the write-miss cache lines have been invalidated, the address ofthe cache line may be sent to replacement policy logic 722 each writecycle. Cache line address which are available may be fed into mux 724 ina replacement address line 726. An address generation line 728 mayinclude an address assigned to a received write instruction. If hit/misslogic 730 determines that the address of the received write instructionis a hit, the address is already stored in a line of the victim cacheand data from the write instruction is merged to the line. If hit/misslogic 730 determines that the address of the received write instructionis a miss, an address from the replacement address line 728 may be usedto write the write-miss command to the victim cache/write miss buffer.An eviction process may be triggered if there are no empty cache linesin the victim cache/write miss buffer to allocate the write miss. In aneviction process, a centralized controller may detect that a write misshas been received that needs to be stored into the victim cache. Thecentralized controller may instruct the victim cache controller 734 toensure that there is a cache line available. This cache line may bebased on a location provided by either the replacement address 726, orthe address generation 728. If there is already a write miss stored inthe identified location, the write miss may be evicted.

FIG. 8 is a block diagram of a MESI RAM 800, in accordance with aspectsof the present disclosure. The MESI RAM 800 may be a memory (e.g., SRAM)or set of registers (e.g., register file), used to track the MESI statusof a corresponding cache line. As an example, the victim cache, incertain cases, may track MESI states in MESI state bits 802 in the MESIRAM 800 for cache entries as a part of writing data. In certain cases,the MESI RAM 800 includes a line corresponding to each line of thecorresponding cache, here, the victim cache. It may be understood thatthe main cache may include a corresponding MESI RAM. In certain cases, asingle MESI RAM may be shared by both a main and victim cache. The MESIRAM 800 may have a set of bits for tracking the MESI state of each cacheline stored in the victim cache. In certain cases, the MESI RAM 800 maybe expanded with additional bits to track other states of cache lines inthe victim cache. The MESI RAM 800 may be expanded to track additionalstates by including additional columns for the additional states. Inthis example, the MESI RAM has been expanded to include state bits in abit field to track a privilege level 804 for each cache line, a red tag806, a blue tag 808, whether a line is valid 810, and whether a line isa write miss buffer or victim cache line 812. It may be noted that whileshown as a part of the MESI RAM, the state bits may be placed withinother memories in other embodiments, such as the tag RAM 704, or in aseparate memory altogether. It may be understood that the MESI RAM maybe configured to track additional states not shown here.

In certain cases, it may be useful to flush all write-miss bufferentries of a certain type stored in the victim cache. As discussedbelow, entries in the write-miss buffer may be associated with a “color”tag. This color tag may be represented by one or more bits such thatwhen a bit is set for a write-miss buffer entry, the color associatedwith the write-miss buffer entry may be “blue” and when the bit is notset, or another bit is set, the write-miss buffer entry may be “red.”Tagging helps allow certain write-miss buffer entries to be associatedwith a particular process running on a processor, while other write-missbuffer entries may be associated with another process running on theprocessor. In certain cases, a processor may want to flush allwrite-miss buffer entries associated with a particular process (e.g., aselective fence operation), but not the write-miss buffer entriesassociated with another process.

To perform a selective fence operation, a processor may assert a drainsignal. Along with the asserted drain signal, the processor may providean indication that a particular tag should be prioritized for draining,either the red or blue tag. Referring back to FIG. 7, the drain signalis passed to the eviction controller 714. The eviction controller 714may check an availability of a higher level cache or memory to seewhether the higher level cache or memory is busy. If the higher levelcache or memory is unavailable, the eviction controller 714 may stallthe processor. If the higher level cache or memory is available, thenthe eviction controller 714 may send an eviction trigger requestindicating the prioritized tag to the victim cache controller 734. Thevictim cache controller 734 may then check with the store queue to seeif there are any outstanding entries in the store queue for any bank ofthe victim cache RAM 706. The victim cache controller 734 may alsoindicate to the store queue 710 to escalate the priority of draining thestore queue. If there is data, for example in the store queue, that hasnot yet been stored to the victim cache, the eviction controller 714 maywait for the store queue 710 to complete. Once the store queue 710 isemptied, the eviction controller 714 indicates to a victim cachecontroller to evict the selected cache lines. The victim cachecontroller 734 sets up the creation of store miss packets to be sent tohigher level caches or other memories. These store miss packets mayinclude the corresponding memory address stored in the victim cache tagRAM 704, byte enable logic in the byte enable memory 702, and the datain the victim cache RAM 706. The victim cache can then send the storemiss packet, for example to a cache controller of a higher level cacheor other memory. When flushing write-miss entries, the write miss packetmay be sent out to other memories and the corresponding entries in thevictim cache RAM 706, victim cache tag RAM 704, and byte enable memory702 may be invalidated. The write miss packet may be placed in a drainbuffer 716 temporarily until the write miss packet is accepted (e.g.,acknowledged) by the other memory. This flushing continues until theprocessor ceases to assert the drain signal. If the processor stopsasserting the drain signal before all write-miss buffer entries of theindicated color have been drained, draining of the write-miss bufferstops wherever in the draining process the victim cache is in when thedrain signal stops being asserted. Once all write miss buffer entries ofthe indicated color tag have been drained, an idle signal is assertedback to the processor. If the processor continues to assert the drainsignal, other write miss buffer entries associated with the other colortag may be drained.

Placing write-misses in a cache system helps provide an opportunity tomerge additional write misses to the write miss already in the cachesystem. As write-misses can stay longer inside the cache, the cache hasan opportunity to merge any future write-misses to write-misses alreadyexisting in the cache system, thus reducing the write-miss traffic outto higher level memories. Reducing the write-misses being sent to higherlevel memories can reduce the pressure on those memories and helpimprove overall performance.

A write merge may occur when a processor issues a second store or writecommand to a particular address at block 712. Where the victim cache isfully associative, the address of the second write may be checked, bythe hit/miss logic 730, against a set of memory addresses stored in eachlocation in the victim cache tag RAM 704. In parallel, a replacementpolicy 722 logic determines a location inside of the victim cache wherethe write transaction could be allocated in case of a write miss.Generally, when a write miss is stored in the victim cache, the writemiss is allocated an entire cache line, even if the data in the writemiss does not need an entire cache line. Thus, a set of memory addressesmay be associated with each write miss stored in the victim cache,including the memory address in the write miss, along with a set ofmemory addresses either before, after, or both before and after thememory address in the write miss. If the address of the second write isdetermined to be within a set of memory addresses associated with awrite-miss stored in the victim cache, then a cache hit is detected andthe hit/miss logic returns the location in the victim cache where thewrite-miss is currently stored. In the E3 pipestage, the color type andprivilege level associated with the second write is merged, for exampleby the victim cache controller 734, with the currently stored color typeand privilege level and the MESI RAM 732 is updated. In certain cases,merging the privilege level may be performed with reference to aprivilege merge table 736. The data of the second write is also mergedinto the corresponding line of the victim cache RAM 706 and thecorresponding entry of the byte enable memory 702 updated based on thebyte enable logic associated with the second write, for example by thevictim cache controller 734. In certain cases, the store queue 710 mayalso include write merge functionality. For example, when back to backwrites are issued to the same address, the store queue 710 mayinternally merge them to help minimize a number of writes to issue tothe victim cache RAM 706.

In certain cases, write merging may be performed across privilegelevels. Merging the privilege level may be performed when performingwrite merging. For example, a first write-miss stored in the victimcache may be associated with a first privilege level. A second writeassociated with a second privilege level may be merged with the storedfirst write-miss. To perform the merge, the privilege levels should bemerged as well. In certain cases, the privilege levels may be merged bymerging the privilege levels to the lower level of the privilege levels.For example, in an embodiment, a write operation may be associated witha two part privilege level such as root/guest-user/supervisor. A firstwrite-miss may be received with a privilege level of root-user. Thisfirst write-miss may be written to the write miss buffer in the victimcache. A second write with privilege level of guest-supervisor may bereceived, the second write directed to an address within the set ofaddresses associated with the first write-miss. When merging theprivilege levels of the two writes, the lower privilege level may beused, resulting in a merged privilege level of guest-user. In certaincases, merging the privilege levels may operate in the same way whenmerging is performed for multiple datapaths. In certain cases, privilegelevel merging may be encoded in the privilege merge table 736,indicating how different privilege levels may be merged. In certaincases, the privilege merge table 736 may be predefined. An example of apredefined privilege merge table 736 is shown in Table 1. In certaincases, the privilege merge table 736 may be configurable.

TABLE 1 Privilege of Write New Privilege Miss to be merged into CurrentPrivilege level of Write Miss Write Miss Buffer on of Write Miss Bufferline after Write Miss Buffer Hit Buffer Hit line transaction mergeGuest-User Guest-User Guest-User Guest-User Guest-Supervisor Guest-UserGuest-User Root-User Guest-User Guest-User Root-Supervisor Guest-UserGuest-Supervisor Guest-User Guest-User Guest-Supervisor Guest-SupervisorGuest-Supervisor Guest-Supervisor Root-User Guest-User Guest-SupervisorRoot-Supervisor Guest-Supervisor Root-User Guest-User Guest-UserRoot-User Guest-Supervisor Guest-User Root-User Root-User Root-UserRoot-User Root-Supervisor Root-User Root-Supervisor Guest-UserGuest-User Root-Supervisor Guest-Supervisor Guest-SupervisorRoot-Supervisor Root-User Root-User Root-Supervisor Root-SupervisorRoot-Supervisor

In certain cases, write merging may be performed for color tags. Forexample, a first write-miss stored in the victim cache may be associatedwith a first color tag, such as red. With reference to FIG. 8, the firstwrite-miss may be associated with an entry in the MESI RAM 800 with abit set in the red tag 806 of a bit field. A second write associatedwith a second color tag may be merged with the stored first write-miss.To perform the merge, the color tags should be merged as well. Incertain cases, the color tags may be merged by combining the color tagsof the writes. For example, if both writes are red tagged, then themerged write remains red tagged. Similarly, the merged write would beblue tagged if both writes are blue tagged. If one write is red taggedand the other write is blue tagged, then the merged write is tagged bothred and blue. Thus, if the second write to be merged is associated witha blue tag, the merged write may have a bit set in the red tag 806 ofthe bit field and a bit set in the blue tag 808 of the bit field. Incertain cases, if a drain operation is then performed, for example forblue tags, the merged write would be drained. Similarly, if a drainoperation is performed for red tags, the merged write would also bedrained as the merged write is tagged as both red and blue.

FIG. 9 is an architectural block diagram of a cache system 900 includinga write-miss buffer, in accordance with aspects of the presentdisclosure. As shown, a cache 902 may receive cache requests from aprocessor 904 via a processor interface 906. In certain cases, the cache902 may be configured to handle either scalar or vector data, or both.The cache 902 is logically divided into parallel sub-caches including amain cache 908 and a victim cache 910. While a single main cache 908 andvictim cache 910 are shown, it may be understood that certainembodiments can include multiple main caches 908 and victim caches 910pairs. In certain cases, there may be a different number of main caches908 and victim caches 910. With respect to the main cache 908, a maincache tag RAM access controller 912 accesses and checks a main cache tagRAM 914 against memory addresses included in the cache requests todetermine whether a memory address is stored in the main cache 908. Amain cache controller 916 directs the appropriate allocation, eviction,arbitration, merge, etc. processes and reads/writes into the main cachememory 918. A main cache store queue structure 920, as described inconjunction with FIG. 3, provides features such as RMW, ECC, writemerging, etc.

The overall architecture of the victim cache 910 is similar to that ofmain cache 908. For example, for a received memory request, a victimcache tag RAM access controller 922 accesses and checks a victim cachetag RAM 924 against memory addresses included in the cache requests todetermine whether a memory address is stored in the victim cache 910. Avictim cache controller 926 similarly directs the appropriateallocation, eviction, arbitration, merge, etc. processes andreads/writes into the victim cache memory 928. A victim cache storequeue 930 provides RMW, ECC, and other features, as discussed inconjunction with FIG. 3. Victim cache logic 932 helps coordinateoperations as between the victim cache 910 and the main cache 908. Itmay be understood that while the victim cache logic 932 is shownseparately, the victim cache logic 932 may be integrated with or a partof other elements, such as the victim cache controller 926. In certaincases, the victim cache 908 includes certain elements which help enablethe victim cache 908 to support write-miss buffer functionality. Forexample, auto flush logic 934 may monitor the processor interface 906and direct draining the portions of the victim cache memory 928allocated for write-miss buffering, for example, to higher level cachesand memories via one or more memory interfaces 936. As another example,MESI RAM 938 or the victim cache tag RAM may be configured to includeone or more bit fields to store state information regarding write-missbuffer entries stored in the victim cache 910.

In certain cases, the victim cache and main cache may include multipledatapaths while configured as parallel caches which appear as a singlecache to a processor. Each datapath is configured to handle a particularcommand in parallel with the other datapaths. For example, a firstdatapath 910 may be handling a read command to the main cache while thesecond datapath 942 handles a write-miss to the victim cache. In anembodiment with two datapaths for the victim cache there may be fourcombinations of cache commands for a first datapath 940 and a seconddatapath 942: read/read, read/write, write/read, and write/write. Inaddition, each command combination could result in a cache hit/hit,hit/miss, miss/hit, or miss/miss. In addition, the two commands couldpotentially be addressed to the same memory address with varyingprivilege levels and color tags. In certain cases, to facilitatecontrolling the parallel caches, a centralized controller 944 may helpcoordinate handling of read and write requests as between the main cache908 and the victim cache 910. In certain cases, the centralizedcontroller 944 may be configured to handle coordination as between themain cache 908 and the victim cache 910 and handing off tasks for themain cache 908 and the victim cache 910 to the main cache controller 916and victim cache controller 926, respectively. The centralizedcontroller 944 may also detect and handle potential conflicts as well.For example, where read request received from a processor, thecentralized controller 944 may check with the main cache 908 and thevictim cache 910 to determine that the addressed memory location of theread request is not stored in either the main cache 908 nor the victimcache 910 and thus is a read miss. The centralized controller 944 mayalso check the victim cache 910 to determine that the addressed memorylocation is not stored as a write-miss in the victim cache 910, or isbeing flushed out of the victim cache 910. If the addressed memorylocation is stored as a write-miss in the victim cache 910, then thecentralized controller 944 may stall the read request to flush theaddressed memory location. The centralized controller 944 may then issuea request to a higher level cache or memory for the addressed memorylocation.

In certain cases, if a read or a write request is a hit in the maincache 908, the results of the victim cache 910 may be disregarded. Wherethe main cache 908 is configured as a one way associative cache, cacherequest conflicts as with multiple datapaths are straight forward. Cacherequest conflicts from the multiple datapaths may be an issue for in thevictim cache 910 where the victim cache 910 is fully associative and canstore write-miss entries as well as cache entries. Cache requestconflicts for two datapaths may be handled as described in the followingexamples. Of note, in the following examples cache operations aredescribed in the context of a particular datapath. However, it may beunderstood that the examples are directed to combinations of particularcache operations, and which datapath a particular cache operation is ondoes not matter.

In a first example, there is a cache read miss for the first datapathand a cache write hit for the second datapath, with two different memoryaddresses. In certain cases where the victim cache 910 is fullyassociative, cache requests can go to any location within the victimcache 910. Referring to FIG. 7, address generation 728 for a locationwithin the victim cache occurs in the E2 pipestage along with addresslookup in the victim cache tag RAM 704, thus address generation 728 fora location within the victim cache is performed before it is knownwhether an address of the cache request is a hit or miss. Oncecentralized controller 944 determines that there is a read miss on themain cache 908, a request to obtain the memory address of the readrequest is issued to a higher level cache or memory. Where the maincache 908 is one way associative, the memory address of the read requestof the first datapath maps to a single location in the main cache 908.If there is already cached data in the single location, the alreadycached data is evicted from the main cache 908 to the victim cache 910to the pre-generated location within the victim cache 910. If thispre-generated location is the same location the cache write of thesecond datapath is a hit on, a conflict occurs. This conflict may bedetected by the centralized controller 944. As indicated above, addressgeneration for a location within the victim cache 910 occurs before itis known whether the address of cache request is a hit or a miss, thusthere is an address generated for a second location within the victimcache 910 for the cache write of the second datapath prior to thedetermination that the cache write is a hit. Based on the detection ofthe conflict, this second location within the victim cache may be usedto store the data evicted from the main cache 908 by the read miss.

In another example, there may be a cache read miss for the firstdatapath and a cache write hit for the second datapath, as with thefirst example, but in this example, the cache read misses in the maincache and the cache write hits in the main cache as well. In certaincases, the main cache is a one-way associate cache (e.g., direct mappedcache) where particular memory addresses are mapped to particularlocations within the main cache. In this example with a direct mappedcache, the cache read and the cache write are mapped to the samelocation in the main cache. In this example, due to the read miss, arequest to obtain the memory address of the read request is issued to ahigher level cache or memory and the already cached data is evicted fromthe main cache 908 to the victim cache 910 to a pre-generated location,here location A, within the victim cache 910. In this example, the cachewrite of the second datapath hits the same location in the main cachewhich is being evicted. One possible solution to such a conflict is toload the requested read miss from the higher level cache or memorydirectly to the victim cache in a separate location. Another solution tothe conflict is for the centralized controller 944 to stall the readmiss such that the already cached data is not evicted to the victimcache 910 until after the cache write of the second datapath completesto location A in the main cache and location A is then evicted to thevictim cache. In certain cases, the stall may occur while waiting forthe memory write and eviction to complete.

As another example, a first cache write may be received on the firstdatapath for the victim cache 910 and a second cache write is alsoreceived on the second datapath for the victim cache 910. If the firstcache write and the second cache write are addressed to different memoryaddresses, then both cache writes may proceed in parallel to the victimcache 910. If the first cache write and the second cache write are bothaddressed to the same memory address and both are cache misses, then thevictim cache controller 926 allocates a single location in the victimcache write-miss memory 928 and merges the first cache write and thesecond cache write in the victim cache store queue 930. The merged datamay then be written to the victim cache write-miss memory 928 along withmerged byte enable, privilege, and color tag information. If the firstcache write and the second cache write are both addressed to the samememory address and both are cache hits, both the first cache write andthe second cache write are merged into the cache at the hit location.

As another example a cache read may be received on the first datapathfor the victim cache 910 and a cache write may be received on the seconddatapath for the victim cache 910. In the cases where the cache read isa hit and the cache write is a hit to different locations in the victimcache 910, the cache read and cache write proceed in parallel withoutconflicts. In the case the cache read is a hit and the cache write is amiss to different locations in the victim cache 910, the cache read andcache write also proceed in parallel without conflicts. In the case thecache read is a miss and the cache write is a hit to different locationsin the victim cache 910, the cache read may use an address generated fora location within the victim cache 910 for the cache write as discussedabove. In the case the cache read is a miss and the cache write is amiss to different locations in the victim cache 910, both the cache readand the cache write use addresses generated for locations within thevictim cache 910.

In another example, a cache read may be received on the first datapathfor the main cache 908 and a cache write may be received on the seconddatapath for the main cache 908. In certain cases, the cache read may bea hit for a location (e.g., cache line or slot) of the main cache 908.The cache write may also be a hit for the same location of the maincache 908. In such a case, the cache read and the cache write mayproceed in parallel without conflicts as the cache read may be performedrelatively quickly and conclude before the cache write needs to accessmain cache location as the cache write proceeds through the store queue.In another case, the cache read may be a hit for a location in the maincache 908. The cache write may be a miss for the same location in themain cache 908. In such a case, the cache commands proceed in parallelnormally as the cache read may complete prior to the cache write needingto access the cache location. In another case, the cache read may be amiss for a location in the main cache 908. The cache write may be a hitfor the same location in the main cache 910. In such a case, as similarto that discussed above, the cache read may be stalled until after thecache write of the second datapath completes to the location in the maincache 908 and is evicted to the victim cache 908. The cache read thenproceeds to read the set of addresses from the higher level cache ormemory into the main cache 910. In another case, the cache read may be amiss for a first location in the main cache 908. The cache write mayalso be a miss for the same location in the main cache 908. In such acase, the cache read and the cache write may proceed in parallel withoutconflicts.

In another example, a cache read may be received on the first datapathfor the victim cache 910 and a cache write may be received on the seconddatapath for the victim cache 910. In certain cases, the cache read maybe a hit for an address stored in the victim cache 910. The cache writemay also be a hit for the same address stored in the victim cache 910.In such a case, the cache read may proceed first and the cache write maybe stalled until after the cache read completes. Alternatively, theorder of the cache write and cache read may be based on the datapath onwhich the cache write and cache read are received, with the cachecommand arriving on a lower (or higher) numbered datapath beingcompleted before the other cache command. In certain cases, the cacheread may be a miss for an address stored in the victim cache 910. Thecache write may also be a miss for the same address stored in the victimcache 910. In such a case, the cache write command may be forwarded to ahigher level cache or memory and then the cache read may obtain the datafrom the higher level cache or memory after the cache write commandcompletes for storage into the victim cache 910.

In another example, a first cache read may be received on the firstdatapath for the victim cache 910 and a second cache read may bereceived on the second datapath for the victim cache 910. If the firstcache read and the second cache read are for different memory addresses,then there are no conflicts for either hits or misses. In certain cases,the first cache read may be a miss for a first address of a set ofaddresses. The second cache read may also be a miss for a second addressof the same set of addresses. If the first cache read and the secondcache read have different priority levels, a higher level cache ormemory is accessed based on the higher of the different priority levels.Otherwise, the higher level cache or memory is accessed and the set ofmemory addresses obtained for storage in the victim cache 910. The casewhere the first cache read and the second cache read are for the sameaddress is handled identically.

FIG. 10 is a flow diagram illustrating a technique for caching data1000, in accordance with aspects of the present disclosure. At block1002, a caching system receives a write memory request for a memoryaddress. For example, a caching system, such as a L1 cache, can receive,via a processor interface, a write request from a processor. At block1004, a first sub-cache of the caching system determines that the memoryaddress is not cached in the first sub-cache. For example, the cachingsystem may include two parallel caches, a main cache and a victim cache.Memory addresses of cache requests may be checked as against both themain cache and victim cache in parallel. A write-miss occurs when awrite request is received for a memory address not contained within thecaching system. At block 1006, second sub-cache of the caching systemdetermines that the memory address is not cached in the secondsub-cache. At block 1008, data associated with the write memory requestis stored in the second sub-cache. For example, the data of the writememory request may be stored in the victim cache. Additionally, theaddress associated with the write memory request may be stored in a tagRAM of the victim cache and byte enable logic associated with the writememory request may be stored in a byte enable memory of the victimcache. At block 1010, an indication that the stored data corresponds toa write-miss is stored in a line type bit of the second sub-cache. Forexample, the victim cache includes a line type bit associated with linesof the victim cache. A bit may be set in the line type bit indicatingthat a particular line of the victim cache includes a write-miss. Atblock 1012, the stored data may be flushed based on the indication. Forexample, stored write-miss data may be flushed the victim cache to ahigher level memory by checking the line type bits of the cache linesand flushing the lines associated with write-misses.

FIG. 11 is a flow diagram illustrating a technique for caching data1100, in accordance with aspects of the present disclosure. At block1102, a caching system receives a write memory request for a memoryaddress. For example, a caching system, such as a L1 cache, can receive,via a processor interface, a write request from a processor. At block1104, a first sub-cache of the caching system determines that the memoryaddress is not cached in the first sub-cache. For example, the cachingsystem may include two parallel caches, a main cache and a victim cache.Memory addresses of cache requests may be checked as against both themain cache and victim cache in parallel. A write-miss occurs when awrite request is received for a memory address not contained within thecaching system. At block 1106, second sub-cache of the caching systemdetermines that the memory address is not cached in the secondsub-cache. At block 1108, data associated with the write memory requestis stored in the second sub-cache. For example, the data of the writememory request may be stored in the victim cache. Additionally, theaddress associated with the write memory request may be stored in a tagRAM of the victim cache and byte enable logic associated with the writememory request may be stored in a byte enable memory of the victimcache. At block 1110, an indication that the stored data corresponds toa write-miss is stored in a line type bit of the second sub-cache. Forexample, the victim cache includes a line type bit associated with linesof the victim cache. A bit may be set in the line type bit indicatingthat a particular line of the victim cache includes a write-miss. Atblock 1112, a line of the second sub-cache storing the write-miss isevicted based on an indication that the line has been fully written. Forexample, a bit may be calculated based on the byte enable logicindicating whether all, or half, of the bytes of a cache line have beenfilled with write data. Cache lines which are filled, or half filled,may be evicted from the victim cache based on the bit.

FIG. 12 is a flow diagram illustrating a technique for caching data1200, in accordance with aspects of the present disclosure. At block1202, a caching system receiving a write memory command for a memoryaddress, the write memory command associated with a first privilegelevel. For example, a caching system, such as a L1 cache, can receive,via a processor interface, a write request from a processor. At block1204, a first sub-cache of the caching system determines that the memoryaddress is not cached in the first sub-cache. For example, the cachingsystem may include two parallel caches, a main cache and a victim cache.Memory addresses of cache requests may be checked as against both themain cache and victim cache in parallel. A write-miss occurs when awrite request is received for a memory address not contained within themain cache nor the victim cache. At block 1206, a second sub-cache ofthe caching system determines that the memory address is not cached inthe second sub-cache, wherein the second sub-cache is configured tostore, in parallel with the first sub-cache, cache data evicted from thefirst sub-cache and write-memory commands that are not cached in thefirst sub-cache. At block 1208, data associated with the write memorycommand is stored in the second sub-cache. At block 1210, a color tagbit associated with the data is stored in the second sub-cache. Forexample, a color tag bit may be stored within a bit field of a memory ofthe second sub-cache, such as the MESI RAM. At block 1212, the storeddata is evicted based on the color tag bit. For example, cache linesstoring write miss data associated with a particular color tag bit maybe evicted while the processor is asserting a drain signal and a colortag priority indication. After all cache lines with the particular colortag bit have been drained, the caching system may assert an idle signal.If the processor continues to assert the drain signal, other cache linesstoring write miss data associated with other color tags may also beevicted.

FIG. 13 is a flow diagram illustrating a technique for caching data1300, in accordance with aspects of the present disclosure. At block1302, a caching system receives a write memory command for a memoryaddress, the write memory command associated with a first color tag. Forexample, a caching system, such as a L1 cache, can receive, via aprocessor interface, a write request from a processor. The write requestmay include in an indication of a color tag associated with the writerequest. At block 1304, a first sub-cache of the caching systemdetermines that the memory address is not cached in the first sub-cache.For example, the caching system may include two parallel caches, a maincache and a victim cache. Memory addresses of cache requests may bechecked as against both the main cache and victim cache in parallel. Awrite-miss occurs when a write request is received for a memory addressnot contained within the caching system. At block 1306, second sub-cacheof the caching system determines that the memory address is not cachedin the second sub-cache. At block 1308, data associated with the writememory command is stored in a cache line of the second sub-cache. Forexample, the data of the write request may be stored in a cache line ofthe victim cache. Additionally, the address associated with the writememory request may be stored in a tag RAM of the victim cache and byteenable logic associated with the write memory request may be stored in abyte enable memory of the victim cache. At block 1310, the first colortag is stored in the second sub-cache. For example, the color tag may bestored using two bits within a bit field of a memory of the secondsub-cache, such as the MESI RAM. At block 1312, a second write memorycommand for the cache line is received, the write memory commandassociated with a second color tag. For example, a second write requestto a memory address targeting the same cache line as the first writerequest may be received. The second write request is associated with asecond color tag. The second color tag may be the same or different fromthe first color tag. At block 1314, the second color tag is merged withthe first color tag. For example, where the first color tag and thesecond color tag represent the same color, merging the two tagscontinues to set that color tag. Where the two color tags are different,merging may set both color tags. At block 1316, the merged color tag isstored. At block 1318, the cache line may be evicted based on the mergedcolor tag. For example, where the first color tag and the second colortag represent the same color, eviction may be based on that color tag.Where the two color tags are different, eviction may be based on eithercolor tag.

FIG. 14 is a flow diagram illustrating a technique for caching data1400, in accordance with aspects of the present disclosure. At block1402, a caching system receiving a first write memory command for amemory address, the first write memory command associated with a firstprivilege level. For example, a caching system, such as a L1 cache, canreceive, via a processor interface, a write request from a processor. Atblock 1404, a first sub-cache of the caching system determines that thememory address is not cached in the first sub-cache. For example, thecaching system may include two parallel caches, a main cache and avictim cache. Memory addresses of cache requests may be checked asagainst both the main cache and victim cache in parallel. A write-missoccurs when a write request is received for a memory address notcontained within the main cache nor the victim cache. At block 1406, asecond sub-cache of the caching system determines that the memoryaddress is not cached in the second sub-cache, wherein the secondsub-cache is configured to store, in parallel with the first sub-cache,cache data evicted from the first sub-cache and write-memory commandsthat are not cached in the first sub-cache. At block 1408, first dataassociated with the first write memory command is stored in a cache lineof the second sub-cache. At block 1410, the first privilege level isstored in the second sub-cache. For example, the first privilege levelmay comprise a two part privilege level, such asroot/guest-user/supervisor. This two part privilege level may be storedusing two bits within a bit field of a memory of the second sub-cache,such as the MESI RAM. At block 1412, a second write memory command forthe cache line is received, the second write memory command associatedwith a second level of privilege. For example, a second write request toa memory address targeting the same cache line as the first writerequest may be received. The second write request is associated with asecond privilege level, which may be the same or different from thefirst write request. At block 1414, the first level of privilege withthe second level of privilege are merged. For example, the privilegelevels may be merged based on a privilege merge table or may be mergedby selecting the lower privilege level of the privilege levels beingmerged. At block 1416, the merged privilege level are stored. At block1418, the merged privilege level is output with the cache line. Forexample, the cache line including a merged first write request andsecond write request may be evicted in a store miss packet including themerged privilege level to a higher level cache or memory.

FIG. 15 is a flow diagram illustrating a technique for caching data1500, in accordance with aspects of the present disclosure. At block1502, a caching system receives a write memory request for a memoryaddress. For example, a caching system, such as a L1 cache, can receive,via a processor interface, a write request from a processor. At block1504, a first sub-cache of the caching system determines that the memoryaddress is not cached in the first sub-cache. For example, the cachingsystem may include two parallel caches, a main cache and a victim cache.Memory addresses of cache requests may be checked as against both themain cache and victim cache in parallel. A write-miss occurs when awrite request is received for a memory address not contained within thecaching system. At block 1506, second sub-cache of the caching systemdetermines that the memory address is not cached in the secondsub-cache. At block 1508, data associated with the write memory requestis stored in the second sub-cache. For example, the data of the writememory request may be stored in the victim cache. Additionally, theaddress associated with the write memory request may be stored in a tagRAM of the victim cache and byte enable logic associated with the writememory request may be stored in a byte enable memory of the victimcache. At block 1510, an indication that the stored data corresponds toa write-miss is stored in a line type bit of the second sub-cache. Forexample, the victim cache includes a line type bit associated with linesof the victim cache. A bit may be set in the line type bit indicatingthat a particular line of the victim cache includes a write-miss.

FIG. 16 is a flow diagram illustrating a technique for caching data1600, in accordance with aspects of the present disclosure. At block1602, a caching system receives two or more cache commands. For example,a caching system, such as a L1 cache, can receive, via a processorinterface, a multiple cache commands using, for example, two or moredata paths. At block 1604, a conflict is determined to exist between thetwo or more cache commands. For example, a cache controller maydetermine that a cache conflict exists as between the two cachecommands. A conflict can arise when the two cache commands causeconflicting changes to the same cache location. At block 1606, aconflict resolution is determined between the received two or more cachecommands. At block 1608, two or more cache commands are sent to a firstsub-cache and a second sub-cache, wherein the second sub-cache isconfigured to store, in parallel with the first sub-cache, cache dataevicted from the first sub-cache and write-memory commands that are notcached in the first sub-cache.

EXAMPLES

In the following section, further exemplary embodiments are provided.

Example 1 includes a caching system comprising: a first sub-cache; and asecond sub-cache in parallel with the first sub-cache, wherein thesecond sub-cache includes: a set of cache lines; line type bitsconfigured to store an indication that a corresponding cache line of theset of cache lines is configured to store write-miss data; and aneviction controller configured to flush stored write-miss data based onthe line type bits.

Example 2 includes the caching system of example 1, wherein the secondsub-cache further comprises: a tag random access memory (RAM) configuredto store a memory address associated with the stored write-miss data;and a byte enable memory configured to store byte enable logicassociated with the stored write-miss data, and wherein the evictioncontroller is configured to flush stored write-miss data by transmittinga memory write request to a higher level memory based the stored memoryaddress and stored byte enable logic.

Example 3 includes the caching system of example 1, wherein the cachingsystem further includes a processor interface, and wherein the evictioncontroller is further configured to: monitor activity on the processorinterface; and flush stored write-miss data based on the monitoredactivity.

Example 4 includes the caching system of example 3, wherein monitoringactivity on the processor interface comprises detecting idle activity onthe processor interface.

Example 5 includes the caching system of example 4, wherein detectingidle activity comprises determining whether memory requests have beenreceived for a predetermined number of clock cycles.

Example 6 includes the caching system of example 4, wherein the evictioncontroller is further configured to: determine that activity on theprocessor interface has resumed; and cancel flushing of the storedwrite-miss data.

Example 7 includes the caching system of example 6, wherein determiningthat activity on the processor interface has resumed comprises detectinga cache request receipt from the processor.

Example 8 includes a method for caching data, comprising: receiving, bya caching system, a write memory request for a memory address;determining, by a first sub-cache of the caching system, that the memoryaddress is not cached in the first sub-cache; determining, by secondsub-cache of the caching system, that the memory address is not cachedin the second sub-cache; storing data associated with the write memoryrequest in the second sub-cache; storing, in a line type bit of thesecond sub-cache, an indication that the stored data corresponds to awrite-miss; and flushing the stored data based on the indication.

Example 9 includes the method of example 8, wherein flushing the storeddata comprises transmitting, by the second-sub cache, a memory writerequest to a higher level memory.

Example 10 includes the method of example 8, further comprisingmonitoring activity on a processer interface, and wherein flushing thestored data is further based on the monitored activity.

Example 11 includes the method of example 10, wherein monitoringactivity on the processor interface comprises detecting idle activity onthe processor interface.

Example 12 includes the method of example 11, wherein detecting idleactivity comprises determining that memory requests have not beenreceived for a predetermined number of clock cycles.

Example 13 includes the method of example 11, further comprising:determining that activity on the processor interface has resumed; andcancelling flushing of the stored write-miss data.

Example 14 includes the method of example 13, wherein determining thatactivity on the processor interface has resumed comprises detecting acache request receipt.

Example 15 includes a device comprising: a processor; a first sub-cache;and a second sub-cache in parallel with the first sub-cache, wherein thesecond sub-cache includes: a set of cache lines; line type bitsconfigured store an indication that a corresponding cache line of theset of cache lines is configured to store write-miss data; and aneviction controller configured to flush stored write-miss data based onthe line type bits.

Example 16 includes the device of example 15, wherein the secondsub-cache further comprises: a tag random access memory (RAM) configuredto store a memory address associated with the stored write-miss data;and a byte enable memory configured to store byte enable logicassociated with the stored write-miss data, and wherein the evictioncontroller is configured to flush stored write-miss data by transmittinga memory write request to a higher level memory based the stored memoryaddress and stored byte enable logic.

Example 17 includes device of example 15, wherein the caching systemfurther includes a processor interface, and wherein the evictioncontroller is further configured to: monitor activity on the processorinterface; and flush stored write-miss data based on the monitoredactivity.

Example 18 includes the device of example 17, wherein monitoringactivity on the processor interface comprises detecting idle activity onthe processor interface.

Example 19 includes the device of example 18, wherein detecting idleactivity comprises determining whether memory requests have beenreceived for a predetermined number of clock cycles.

Example 20 includes the device of example 18, wherein the evictioncontroller is further configured to: determine that activity on theprocessor interface has resumed; and cancel flushing of the storedwrite-miss data.

Example 21 includes a caching system comprising: a first sub-cache; anda second sub-cache, coupled in parallel with the first cache, forstoring cache data evicted from the first sub-cache and write-memorycommands that are not cached in the first sub-cache, and wherein thesecond sub-cache includes: color tag bits configured to store anindication that a corresponding cache line of the second sub-cachestoring write miss data is associated with a color tag, and an evictioncontroller configured to evict cache lines of the second sub-cachestoring write-miss data based on the color tag associated with the cacheline.

Example 22 includes the caching system of Example 21, wherein the secondsub-cache includes a bit field configured to store a state of the colortags.

Example 23 includes the caching system of Example 22, wherein the bitfield includes a bit storing the state for each color tag.

Example 24 includes the caching system of Example 23, wherein the bitfield is stored in a memory tracking a modified, exclusive, shared, andinvalid (MESI) state of entries in the second sub-cache.

Example 25 includes the caching system of Example 24, wherein the MESImemory is configured to store an indication that the corresponding cacheline of the second sub-cache is storing write-miss data.

Example 26 includes the caching system of Example 21, wherein thecaching system further includes a processor interface, and wherein theeviction controller is further configured to: receive, on the processorinterface, a color tag priority indication; and evict cache lines of thesecond sub-cache storing write-miss data based on the received color tagpriority indication.

Example 27 includes the caching system of claim 6, wherein the evictioncontroller is further configured to assert an idle signal on theprocessor interface after all cache lines of the second sub-cachestoring write-miss data with color tag bits corresponding to the colortag priority indication have been evicted.

Example 28 includes a method for caching data, comprising receiving, bya caching system, a write memory command for a memory address;determining, by a first sub-cache of the caching system, that the memoryaddress is not cached in the first sub-cache; determining, by secondsub-cache of the caching system, that the memory address is not cachedin the second sub-cache, wherein the second sub-cache is configured tostore, in parallel with the first sub-cache, cache data evicted from thefirst sub-cache and write-memory commands that are not cached in thefirst sub-cache; storing data associated with the write memory commandin the second sub-cache; storing, in the second sub-cache, a color tagbit associated with the data; and evicting the stored data based on thecolor tag bit.

Example 29 includes the method of Example 28, further comprising:receiving a color tag priority indication from a processor; receiving adrain signal from the processor; and wherein evicting the stored data isbased on the received color tag priority indication.

Example 30 includes the method of claim Example 29, further comprising:asserting an idle signal after all cache lines of the second sub-cachestoring write-miss data with color tag bits corresponding to the colortag priority indication have been evicted.

Example 31 includes the method of Example 30, further comprising:continuing to receive the drain signal from the processor; and evictingstored data with color tag bits which do not correspond to the color tagpriority indication.

Example 32 includes the method of Example 39, further comprisingdetermining that storing the data has not been completed; and waitinguntil the data has been stored,

Example 33 includes the method of Example 38, further comprising storingthe color tag bit in a bit field associated with a corresponding colortag bit.

Example 34 includes a device comprising: a processor; a first sub-cache;and a second sub-cache, coupled in parallel with the first sub-cache,for storing cache data evicted from the first sub-cache and write-memorycommands that are not cached in the first sub-cache, and wherein thesecond sub-cache includes: color tag bits configured to store anindication that a corresponding cache line of the second sub-cachestoring write-miss data is associated with a color tag, and an evictioncontroller configured to evict the cache line of the second sub-cachestoring write-miss data based on the color tag associated with the cacheline.

Example 35 includes the device of Example 34, wherein the secondsub-cache includes a bit field configured to store a state of the colortags.

Example 36 includes the device of Example 35, wherein the bit fieldincludes a bit storing the state for each color tag.

Example 37 includes the device of Example 36, wherein the bit field isstored in a memory tracking a modified, exclusive, shared, and invalid(MESI) state of entries in the second sub-cache.

Example 38 includes the device of Example 37, wherein the MESI memory isconfigured to store an indication that the corresponding cache line ofthe second sub-cache is storing write-miss data.

Example 39 includes the device of Example 34, wherein the caching systemfurther includes a processor interface, and wherein the evictioncontroller is further configured to: receive, on the processorinterface, a color tag priority indication; and evict cache lines of thesecond sub-cache storing write-miss data based on the received color tagpriority indication.

Example 40 includes the device of Example 39, wherein the evictioncontroller is further configured to assert an idle signal on theprocessor interface after all cache lines of the second sub-cachestoring write-miss data with color tag bits corresponding to the colortag priority indication have been evicted.

Example 41 includes a caching system comprising: a first sub-cache; anda second sub-cache, coupled in parallel with the first sub-cache, forstoring cache data evicted from the first sub-cache and write-memorycommands that are not cached in the first sub-cache, the secondsub-cache including: color tag bits configured to store an indicationthat a corresponding line of the second sub-cache is associated with acolor tag; and an eviction controller configured to evict cache lines ofthe second sub-cache storing write-memory data based on the color tagassociated with the line, and wherein the second sub-cache is furtherconfigured to: receive a first write memory command for a memoryaddress, the write memory command associated with a first color tag;store first data associated with the first write memory command in acache line of the second sub-cache; store the first color tag in thesecond sub-cache; receive a second write memory command for the cacheline, the write memory command associated with a second color tag; mergethe second color tag with the first color tag; store the merged colortag; and evict the cache line based on the merged color tag.

Example 42 includes the caching system of Example 41, wherein the firstcolor tag and the second color tag represent the same color tag.

Example 43 includes the caching system of Example 41, wherein the firstcolor tag and the second color tag represent different color tags, andwherein the merged color tag comprises a first bit representing thefirst color tag and a second bit representing the second color tag.

Example 44 includes the caching system of Example 43, wherein the secondsub-cache is further configured to evict the cache line based on thefirst color tag.

Example 45 includes the caching system of Example 44, wherein the secondsub-cache is further configured to evict the cache line based on thesecond color tag.

Example 46 includes the caching system of Example 41, wherein the firstcolor tag and second color tag are stored in a bit field in a memorytracking modified, exclusive, shared, and invalid (MESI) state ofentries in the second sub-cache data.

Example 47 includes the caching system of Example 46, wherein the bitfield includes a bit storing the state for each color tag.

Example 48 includes a method for caching data, comprising: receiving, bya caching system, a write memory command for a memory address, the writememory command associated with a first color tag; determining, by afirst sub-cache of the caching system, that the memory address is notcached in the first sub-cache; determining, by second sub-cache of thecaching system, that the memory address is not cached in the secondsub-cache, wherein the second sub-cache is configured to store, inparallel with the first sub-cache, cache data evicted from the firstsub-cache and write-memory commands that are not cached in the firstsub-cache; storing first data associated with the first write memorycommand in a cache line of the second sub-cache; storing the first colortag in the second sub-cache; receiving a second write memory command forthe cache line, the write memory command associated with a second colortag; merging the second color tag with the first color tag; storing themerged color tag; and evicting the cache line based on the merged colortag.

Example 49 includes the method of Example 48, wherein the first colortag and the second color tag represent the same color tag.

Example 50 includes the method of Example 48, wherein the first colortag and the second color tag represent different color tags, and whereinthe merged color tag comprises a first bit representing the first colortag and a second bit representing the second color tag.

Example 51 includes the method of Example 50, wherein the evicting isbased on the first color tag.

Example 52 includes the method of Example 51, wherein the evicting isbased on the second color tag.

Example 53 includes the method of Example 48, wherein the first colortag and second color tag are stored in a bit field in a memory trackinga modified, exclusive, shared, and invalid (MESI) state of entries inthe second sub-cache data.

Example 54 includes the method of claim Example 53, wherein the bitfield includes a bit storing the state for each color tag.

Example 55 includes a device comprising: a processor; a first sub-cache;and a second sub-cache coupled in parallel with the first sub-cache, forstoring cache data evicted from the first sub-cache and write-memorycommands that are not cached in the first sub-cache, and the secondsub-cache including: color tag bits configured to store an indicationthat a corresponding line of the second sub-cache is associated with acolor tag; and an eviction controller configured to evict cache lines ofthe second sub-cache storing write-memory data based on the color tagassociated with the line, and wherein the second sub-cache is furtherconfigured to: receive a first write memory command for a memoryaddress, the write memory command associated with a first color tag;store first data associated with the first write memory command in acache line of the second sub-cache; store the first color tag in thesecond sub-cache; receive a second write memory command for the cacheline, the write memory command associated with a second color tag; mergethe second color tag with the first color tag; store the merged colortag; and evict the cache line based on the merged color tag.

Example 56 includes the device of Example 55, wherein the first colortag and the second color tag represent the same color tag.

Example 57 includes the device of Example 55, wherein the first colortag and the second color tag represent different color tags, and whereinthe merged color tag comprises a first bit representing the first colortag and a second bit representing the second color tag.

Example 58 includes the device of Example 57, wherein the secondsub-cache is further configured to evict the cache line based on thefirst color tag.

Example 59 includes the device of Example 58, wherein the secondsub-cache is further configured to evict the cache line data based onthe second color tag.

Example 60 includes the device of Example 55, wherein the first colortag and second color tag are stored in a bit field in a memory trackingmodified, exclusive, shared, and invalid (MESI) state of entries in thesecond sub-cache data.

Example 61 includes a caching system comprising: a first sub-cache; asecond sub-cache, coupled in parallel with the first sub-cache, forstoring cache data evicted from the first sub-cache and write-memorycommands that are not cached in the first sub-cache, the secondsub-cache including privilege bits configured to store an indicationthat a corresponding cache line of the second sub-cache is associatedwith a level of privilege, and wherein the second sub-cache is furtherconfigured to: receive a first write memory command for a memoryaddress, the first write memory command associated with a first level ofprivilege; store, in a cache line of the second sub-cache, first dataassociated with the first write memory command; store, in the secondsub-cache, the level of privilege associated with the cache line;receive a second write memory command for the cache line, the secondwrite memory command associated with a second level of privilege; mergethe first level of privilege with the second level of privilege; storethe merged privilege level;

and output the merged privilege level with the cache line.

Example 62 includes the caching system of Example 61, wherein privilegelevel information is stored in a bit field in a memory trackingmodified, exclusive, shared, and invalid (MESI) state of entries in thesecond sub-cache data.

Example 63 includes the caching system of Example 62, wherein theprivilege level comprises a two part privilege level.

Example 64 includes the caching system of Example 63, wherein the bitfield includes two bits for storing the privilege level.

Example 65 includes the caching system of Example 61, wherein the secondsub-cache includes a privilege merge table, the privilege merge tableindicating how different levels of privilege are merged.

Example 66 includes the caching system of Example 65, wherein theprivilege merge table is predefined.

Example 67 includes the caching system of Example 61, wherein mergingthe first level of privilege with the second level of privilegecomprises selecting the lower privilege level.

Example 68 includes a method for caching data, comprising: receiving, bya caching system, a first write memory command for a memory address, thefirst write memory command associated with a first privilege level;determining, by a first sub-cache of the caching system, that the memoryaddress is not cached in the first sub-cache; determining, by secondsub-cache of the caching system, that the memory address is not cachedin the second sub-cache, wherein the second sub-cache is configured tostore, in parallel with the first sub-cache, cache data evicted from thefirst sub-cache and write-memory commands that are not cached in thefirst sub-cache; storing first data associated with the first writememory command in a cache line of the second sub-cache; storing thefirst privilege level in the second sub-cache; receiving a second writememory command for the cache line, the second write memory commandassociated with a second level of privilege; merging the first level ofprivilege with the second level of privilege; storing the mergedprivilege level; and outputting the merged privilege level with thecache line.

Example 69 includes the method of Example 67, wherein the privilegelevel comprises a two part privilege level.

Example 70 includes the method of Example 69, wherein the secondsub-cache includes two bits in a bit field for storing the privilegelevel.

Example 71 includes the method of Example 68, wherein the merging isbased on a privilege merge table indicating how different levels ofprivilege are merged.

Example 72 includes the method of Example 71, wherein the privilegemerge table is predefined.

Example 73 includes the method of Example 68, wherein merging the firstlevel of privilege with the second level of privilege comprisesselecting the lower privilege level.

Example 74 includes a device comprising: a processor; a first sub-cache;and a second sub-cache coupled in parallel with the first sub-cache, forstoring cache data evicted from the first sub-cache and write-memorycommands that are not cached in the first sub-cache, the secondsub-cache including privilege bits configured to store an indicationthat a corresponding cache line of the second sub-cache is associatedwith a level of privilege, and wherein the second sub-cache is furtherconfigured to: receive a first write memory command for a memoryaddress, the first write memory command associated with a first level ofprivilege; store, in a cache line of the second sub-cache, first dataassociated with the first write memory command; store, in the secondsub-cache, the level of privilege associated with the cache line;receive a second write memory command for the cache line, the secondwrite memory command associated with a second level of privilege; mergethe first level of privilege with the second level of privilege; storethe merged privilege level; and output the merged privilege level withthe cache line.

Example 75 includes the device of Example 74, wherein privilege levelinformation is stored in a bit field in a memory tracking modified,exclusive, shared, and invalid (MESI) state of entries in the secondsub-cache data.

Example 76 includes the device of Example 75, wherein the privilegelevel comprises a two part privilege level.

Example 77 includes the device of Example 76, wherein the bit fieldincludes two bits for storing the privilege level.

Example 78 includes the device of Example 75, wherein the secondsub-cache includes a privilege merge table, the privilege merge tableindicating how different levels of privilege are merged.

Example 79 includes the device of Example 78, wherein the privilegemerge table is predefined.

Example 80 includes the device of Example 75, wherein merging the firstlevel of privilege with the second level of privilege comprisesselecting the lower privilege level.

Example 81 includes a caching system comprising: a first sub-cache; anda second sub-cache coupled in parallel with the first sub-cache; whereinthe second sub-cache includes line type bits configured to store anindication that a corresponding line of the second sub-cache isconfigured to store write-miss data.

Example 82 includes the caching system of Example 81, wherein the secondsub-cache further comprises a tag random access memory (RAM) configuredto store a memory address associated with the stored write-miss data.

Example 83 includes the caching system of Example 81, wherein the secondsub-cache further comprises a byte enable memory configured to storebyte enable logic associated with the stored write-miss data.

Example 84 includes the caching system of Example 83, wherein thebyte-enable memory includes memory positions corresponding with eachcache line of the second sub-cache.

Example 85 includes the caching system of Example 81, wherein the storedindication indicates whether the corresponding cache line of the secondsub-cache is storing write miss entries or cache entries.

Example 86 includes the caching system of Example 81, wherein the linetype bits are stored in a memory tracking modified, exclusive, shared,and invalid (MESI) state of entries in the second sub-cache.

Example 87 includes the caching system of Example 81, wherein the firstsub-cache is a n-way set associative cache and wherein the secondsub-cache is a fully associative cache.

Example 88 includes a method for caching data, comprising: receiving, bya caching system, a write memory request for a memory address;determining, by a first sub-cache of the caching system, that the memoryaddress is not cached in the first sub-cache; determining, by secondsub-cache of the caching system, the second sub-cache coupled inparallel with the first sub-cache, that the memory address is not cachedin the second sub-cache; storing data associated with the write memoryrequest in the second sub-cache; and storing, in a line type bit of thesecond sub-cache, an indication that the stored data corresponds to awrite-miss.

Example 89 includes the method of Example 88, further comprising storinga memory address associated with the stored write-miss data in a tagrandom access memory (RAM).

Example 90 includes the method of Example 88, further comprising storingbyte enable logic associated with the stored write-miss data in abyte-enable memory.

Example 91 includes the method of Example 88, wherein the line type bitsare stored in a memory tracking modified, exclusive, shared, and invalid(MESI) state of entries in the second sub-cache.

Example 92 includes the method of Example 88, further comprising:evicting a cache entry from a line of the second sub-cache, and whereinstoring data associated with the write memory request comprises storingthe data in the line.

Example 93 includes the method of Example 88, wherein the storedindication indicates whether the corresponding cache line of the secondsub-cache is storing write miss entries or cache entries.

Example 94. A device comprising: a processor; a first sub-cache; and asecond sub-cache coupled in parallel with the first sub-cache; whereinthe second sub-cache includes line type bits configured to store anindication that a corresponding line of the second sub-cache isconfigured to store write-miss data.

Example 95 includes the device of Example 94, wherein the secondsub-cache further comprises a tag random access memory (RAM) configuredto store a memory address associated with the stored write-miss data.

Example 96 includes the device of Example 94, wherein the secondsub-cache further comprises a byte enable memory configured to storebyte enable logic associated with the stored write-miss data.

Example 97 includes the device of Example 96, wherein the byte-enablememory includes memory positions corresponding with each cache line ofthe second sub-cache.

Example 98 includes the device of Example 94, wherein the storedindication indicates whether the corresponding cache line of the secondsub-cache is storing write miss entries or cache entries.

Example 99 includes the device of Example 94, wherein the line type bitsare stored in a memory tracking modified, exclusive, shared, and invalid(MESI) state of entries in the second sub-cache.

Example 100 includes the device of claim 94, wherein the first sub-cacheis a n-way set associative cache and wherein the second sub-cache is afully associative cache.

Example 101 includes a caching system comprising: a first sub-cache; anda second sub-cache in parallel with the first sub-cache, wherein thesecond sub-cache includes: a set of cache lines; line type bitsconfigured to store an indication that a corresponding cache line of theset of cache lines is configured to store write-miss data; and aneviction controller configured to evict a cache line of the secondsub-cache storing write-miss data based on an indication that the cacheline has been fully written.

Example 102 includes the caching system of Example 101, wherein theeviction controller is further configured to evict a cache line of thesecond sub-cache storing write-miss data based on an indication thathalf of the cache line has been written.

Example 103 includes the caching system of Example 102, wherein theeviction controller is configured to determine that no cache line of thesecond sub-cache storing write-miss data is fully written beforeevicting the cache line based on the indication that half of the cacheline has been written.

Example 104 includes the caching system of Example 102, wherein thesecond sub-cache further comprises a byte enable memory configured tostore byte enable logic associated with the stored write-miss data, andwherein the byte enable memory includes a first bit storing theindication that the cache line has been fully written.

Example 105 includes the caching system of Example 104, wherein the byteenable memory includes a second bit storing the indication that half ofthe cache line has been written.

Example 106 includes the caching system of Example 105, wherein thefirst and second bits are determined based on the byte enable logicassociated with the stored write-miss data.

Example 107 includes the caching system of Example 106, wherein thefirst and second bits are determined based on the byte enable logic eachtime write-miss data is stored in the second sub-cache.

Example 108 includes a method for caching data, comprising: receiving,by a caching system, a write memory request for a memory address;determining, by a first sub-cache of the caching system, that the memoryaddress is not cached in the first sub-cache; determining, by secondsub-cache of the caching system, that the memory address is not cachedin the second sub-cache; storing data associated with the write memoryrequest in the second sub-cache; storing, in a line type bit of thesecond sub-cache, an indication that the stored data corresponds to awrite-miss; and evicting a cache line of the second sub-cache storingthe write-miss based on an indication that the cache line has been fullywritten.

Example 109 includes the method of Example 108, further comprisingevicting a cache line of the second sub-cache storing write-miss databased on an indication that half of the cache line has been written.

Example 110 includes the method of Example 109, further comprisingdetermining that no cache line of the second sub-cache storingwrite-miss data is fully written before evicting the cache line based onthe indication that half of the cache line has been written.

Example 111 includes the method of Example 109, further comprisingstoring a first bit indicating that the cache line has been fullywritten in a byte enable memory of the second sub-cache.

Example 112 includes the method of Example 111, further comprisingstoring a second bit indicating that the cache line is half written inthe byte enable memory of the second sub-cache.

Example 113 includes the method of Example 112, wherein the first andsecond bits are determined based on the byte enable logic associatedwith the stored write-miss data.

Example 114 includes the method of Example 113, wherein the first andsecond bits are determined based on the byte enable logic each timewrite-miss data is stored in the second sub-cache.

Example 115 includes a device comprising: a processor; a firstsub-cache; and a second sub-cache in parallel with the first sub-cache;wherein the second sub-cache includes: a set of cache lines; line typebits configured to store an indication that a corresponding cache lineof the set of cache lines is configured to store write-miss data, and aneviction controller configured to evict a cache line of the secondsub-cache storing write-miss data based on an indication that the cacheline has been fully written.

Example 116 includes the device of Example 115, wherein the evictioncontroller is further configured to evict a cache line of the secondsub-cache storing write-miss data based on an indication that half ofthe cache line has been written.

Example 117 includes the device of Example 116, wherein the evictioncontroller is configured to determine that no cache line of the secondsub-cache storing write-miss data is fully written before evicting thecache line based on the indication that half of the cache line has beenwritten.

Example 118 includes the device of Example 118, wherein the secondsub-cache further comprises a byte enable memory configured to storebyte enable logic associated with the stored write-miss data, andwherein the byte enable memory includes a first bit storing theindication that the cache line has been fully written.

Example 119 includes the device of Example 118, wherein the byte enablememory includes a second bit storing the indication that half of thecache line has been written.

Example 120 includes the device of Example 119, wherein the first andsecond bits are determined based on the byte enable logic associatedwith the stored write-miss data.

Example 121 includes a caching system comprising: a first sub-cache; asecond sub-cache, coupled in parallel with the first sub-cache, forstoring cache data evicted from the first sub-cache and write-memorycommands that are not cached in the first sub-cache; and a cachecontroller configured to: receive two or more cache commands; determinea conflict exists between the received two or more cache commands;determine a conflict resolution between the received two or more cachecommands; and sending the two or more cache commands to the firstsub-cache and the second sub-cache.

Example 122 includes the caching system of Example 121, wherein thecache command is a read command and wherein the cache controller isfurther configured to: determine, based on a memory address of the readcommand, that the memory address is not cached in the first sub-cache;determine, based on the memory address, that the memory address isstored as a write-memory command in the second sub-cache; and stall theread command until the write-memory command is drained from the secondsub-cache.

Example 123 includes the caching system of Example 121, wherein thecache controller is further configured to receive two or more cachecommands in parallel.

Example 124 includes the caching system of Example 123, wherein the twoor more cache commands include a read command and a write command, andwherein the cache controller is further configured to: determine, basedon a first memory address of the read command, a first cache location inthe first sub-cache; determine that the first memory address is notcached in the first sub-cache at the first cache location; receive asecond cache location in the second sub-cache associated with the readcommand; determine, based on a second memory address of the writecommand, that the second memory address is stored as write-memorycommand in the second sub-cache at the second cache location; receive athird cache location in the second sub-cache associated with the writecommand; and evict a cached memory item from the first cache location inthe first sub-cache to the third cache location in the second sub-cache.

Example 125 includes the caching system of Example 123, wherein the twoor more cache commands include a read command and a write command, andwherein the cache controller is further configured to: determine, basedon a first memory address of the read command, a first cache location inthe first sub-cache; determine that the first memory address is notcached in the first sub-cache at the first cache location; determine,based on a second memory address of the write command, that the secondmemory address is stored at the first location in the first sub-cachestall the read command; perform the write command; evict the performedwrite commend to the second sub-cache; and resume the read command afterthe eviction.

Example 126 includes the caching system of Example 123, wherein the twoor more cache commands include a first write command and a second writecommand, and wherein the cache controller is further configured to:determine that a first memory address of the first write command is notcached in the first sub-cache; determine that a second memory address ofthe second write command is not cached in the first sub-cache; determinethat the first memory address and the second memory address match; mergethe first write command and the second write command; and store themerged commands to the second sub-cache.

Example 127 includes the caching system of Example 123, wherein the twoor more cache commands include a read command and a write command, andwherein the cache controller is further configured to: determine that afirst memory address of the read command is cached in the firstsub-cache; determine that a second memory address of the write commandis cached in the first sub-cache; determine that the first memoryaddress and the second memory address match; stall the write commanduntil the read command is completed; and resume the write command.

Example 128 includes a method for caching data, comprising: receivingtwo or more cache commands; determining a conflict exists between thetwo or more cache commands; determining a conflict resolution betweenthe received two or more cache commands; and sending the two or morecache commands to a first sub-cache and a second sub-cache, wherein thesecond sub-cache is configured to store, in parallel with the firstsub-cache, cache data evicted from the first sub-cache and write-memorycommands that are not cached in the first sub-cache.

Example 129 includes the method of claim 8, wherein the cache command isa read command and further comprising: determining, based on a memoryaddress of the read command, that the memory address is not cached inthe first sub-cache; determining, based on the memory address, that thememory address is stored as a write-memory command in the secondsub-cache; and stalling the read command until the write-memory commandis drained from the second sub-cache.

Example 130 includes the method of Example 128, wherein the two or morecache commands are received in parallel.

Example 131 includes the method of Example 130, wherein the two or morecache commands include a read command and a write command, and furthercomprising: determining, based on a first memory address of the readcommand, a first cache location in the first sub-cache; determining thatthe first memory address is not cached in the first sub-cache at thefirst cache location; receiving a second cache location in the secondsub-cache associated with the read command; determining, based on asecond memory address of the write command, that the second memoryaddress is stored as write-memory command in the second sub-cache at thesecond cache location; receiving a third cache location in the secondsub-cache associated with the write command; evicting a cached memoryitem from the first cache location in the first sub-cache to the thirdcache location in the second sub-cache.

Example 132 includes the method of Example 130, wherein the two or morecache commands include a read command and a write command, and furthercomprising: determining, based on a first memory address of the readcommand, a first cache location in the first sub-cache; determining thatthe first memory address is not cached in the first sub-cache at thefirst cache location; determining, based on a second memory address ofthe write command, that the second memory address is stored at the firstlocation in the first sub-cache; stalling the read command; performingthe write command; evicting the performed write commend to the secondsub-cache; and resuming the read command after the eviction.

Example 133 includes the method of Example 130, wherein the two or morecache commands include a first write command and a second write command,and wherein the cache controller is further configured to: determiningthat a first memory address of the first write command is not cached inthe first sub-cache; determining that a second memory address of thesecond write command is not cached in the first sub-cache; determiningthat the first memory address and the second memory address match;merging the first write command and the second write command; andstoring the merged commands to the second sub-cache.

Example 134 includes the method of Example 130, wherein the two or morecache commands include a read command and a write command, and furthercomprising: determining that a first memory address of the read commandis cached in the first sub-cache; determining that a second memoryaddress of the write command is cached in the first sub-cache;determining that the first memory address and the second memory addressmatch; stalling the write command until the read command is completed;and resuming the write command.

Example 135 includes a device comprising: a processor; a firstsub-cache; and a second sub-cache coupled in parallel with the firstsub-cache, for storing cache data evicted from the first sub-cache andwrite-memory commands that are not cached in the first sub-cache; and acache controller configured to: receive two or more cache commands;determine a conflict exists between the received two or more cachecommands; determine a conflict resolution between the received two ormore cache commands; and sending the two or more cache commands to thefirst sub-cache and the second sub-cache.

Example 136 includes the device of Example 135, wherein the cachecommand is a read command and wherein the cache controller is furtherconfigured to: determine, based on a memory address of the read command,that the memory address is not cached in the first sub-cache; determine,based on the memory address, that the memory address is stored as awrite-memory command in the second sub-cache; and stall the read commanduntil the write-memory command is drained from the second sub-cache.

Example 137 includes the device of Example 135, wherein the cachecontroller is further configured to receive two or more cache commandsin parallel.

Example 138 includes the device of Example 137, wherein the two or morecache commands include a read command and a write command, and whereinthe cache controller is further configured to: determine, based on afirst memory address of the read command, a first cache location in thefirst sub-cache; determine that the first memory address is not cachedin the first sub-cache at the first cache location; receive a secondcache location in the second sub-cache associated with the read command;determine, based on a second memory address of the write command, thatthe second memory address is stored as write-memory command in thesecond sub-cache at the second cache location; receive a third cachelocation in the second sub-cache associated with the write command; andevict a cached memory item from the first cache location in the firstsub-cache to the third cache location in the second sub-cache.

Example 139 includes the device of Example 137, wherein the two or morecache commands include a read command and a write command, and whereinthe cache controller is further configured to: determine, based on afirst memory address of the read command, a first cache location in thefirst sub-cache; determine that the first memory address is not cachedin the first sub-cache at the first cache location; determine, based ona second memory address of the write command, that the second memoryaddress is stored at the first location in the first sub-cache; stallthe read command; perform the write command; evict the performed writecommend to the second sub-cache; and resume the read command after theeviction.

Example 140 includes the device of Example 137, wherein the two or morecache commands include a first write command and a second write command,and wherein the cache controller is further configured to: determinethat a first memory address of the first write command is not cached inthe first sub-cache; determine that a second memory address of thesecond write command is not cached in the first sub-cache; determinethat the first memory address and the second memory address match; mergethe first write command and the second write command; and store themerged commands to the second sub-cache.

The term “couple” is used throughout the specification. The term maycover connections, communications, or signal paths that enable afunctional relationship consistent with the description of the presentdisclosure. For example, if device A generates a signal to controldevice B to perform an action, in a first example device A is coupled todevice B, or in a second example device A is coupled to device B throughintervening component C if intervening component C does notsubstantially alter the functional relationship between device A anddevice B such that device B is controlled by device A via the controlsignal generated by device A.

Modifications are possible in the described embodiments, and otherembodiments are possible, within the scope of the claims.

What is claimed is:
 1. A caching system comprising: a first sub-cache;and a second sub-cache, coupled in parallel with the first cache, forstoring cache data evicted from the first sub-cache and write-memorycommands that are not cached in the first sub-cache, and wherein thesecond sub-cache includes: color tag bits configured to store anindication that a corresponding cache line of the second sub-cachestoring write miss data is associated with a color tag, and an evictioncontroller configured to evict cache lines of the second sub-cachestoring write-miss data based on the color tag associated with the cacheline.
 2. The caching system of claim 1, wherein the second sub-cacheincludes a bit field configured to store a state of the color tags. 3.The caching system of claim 2, wherein the bit field includes a bitstoring the state for each color tag.
 4. The caching system of claim 3,wherein the bit field is stored in a memory tracking a modified,exclusive, shared, and invalid (MESI) state of entries in the secondsub-cache.
 5. The caching system of claim 4, wherein the MESI memory isconfigured to store an indication that the corresponding cache line ofthe second sub-cache is storing write-miss data.
 6. The caching systemof claim 1, wherein the caching system further includes a processorinterface, and wherein the eviction controller is further configured to:receive, on the processor interface, a color tag priority indication;and evict cache lines of the second sub-cache storing write-miss databased on the received color tag priority indication.
 7. The cachingsystem of claim 6, wherein the eviction controller is further configuredto assert an idle signal on the processor interface after all cachelines of the second sub-cache storing write-miss data with color tagbits corresponding to the color tag priority indication have beenevicted.
 8. A method for caching data, comprising: receiving, by acaching system, a write memory command for a memory address;determining, by a first sub-cache of the caching system, that the memoryaddress is not cached in the first sub-cache; determining, by secondsub-cache of the caching system, that the memory address is not cachedin the second sub-cache, wherein the second sub-cache is configured tostore, in parallel with the first sub-cache, cache data evicted from thefirst sub-cache and write-memory commands that are not cached in thefirst sub-cache; storing data associated with the write memory commandin the second sub-cache; storing, in the second sub-cache, a color tagbit associated with the data; and evicting the stored data based on thecolor tag bit.
 9. The method of claim 8, further comprising: receiving acolor tag priority indication from a processor; receiving a drain signalfrom the processor; and wherein evicting the stored data is based on thereceived color tag priority indication.
 10. The method of claim 9,further comprising: asserting an idle signal after all cache lines ofthe second sub-cache storing write-miss data with color tag bitscorresponding to the color tag priority indication have been evicted.11. The method of claim 10, further comprising: continuing to receivethe drain signal from the processor; and evicting stored data with colortag bits which do not correspond to the color tag priority indication.12. The method of claim 9, further comprising: determining that storingthe data has not been completed; and waiting until the data has beenstored,
 13. The method of claim 8, further comprising storing the colortag bit in a bit field associated with a corresponding color tag bit.14. A device comprising: a processor; a first sub-cache; and a secondsub-cache, coupled in parallel with the first sub-cache, for storingcache data evicted from the first sub-cache and write-memory commandsthat are not cached in the first sub-cache, and wherein the secondsub-cache includes: color tag bits configured to store an indicationthat a corresponding cache line of the second sub-cache storingwrite-miss data is associated with a color tag, and an evictioncontroller configured to evict the cache line of the second sub-cachestoring write-miss data based on the color tag associated with the cacheline.
 15. The device of claim 14, wherein the second sub-cache includesa bit field configured to store a state of the color tags.
 16. Thedevice of claim 15, wherein the bit field includes a bit storing thestate for each color tag.
 17. The device of claim 16, wherein the bitfield is stored in a memory tracking a modified, exclusive, shared, andinvalid (MESI) state of entries in the second sub-cache.
 18. The deviceof claim 17, wherein the MESI memory is configured to store anindication that the corresponding cache line of the second sub-cache isstoring write-miss data.
 19. The device of claim 14, wherein the cachingsystem further includes a processor interface, and wherein the evictioncontroller is further configured to: receive, on the processorinterface, a color tag priority indication; and evict cache lines of thesecond sub-cache storing write-miss data based on the received color tagpriority indication.
 20. The device of claim 19, wherein the evictioncontroller is further configured to assert an idle signal on theprocessor interface after all cache lines of the second sub-cachestoring write-miss data with color tag bits corresponding to the colortag priority indication have been evicted.